Skip to content
← All work

2025 · Data Engineering

Azure HDFS Sync

An incremental Azure Blob Storage download pipeline for geophysical simulation outputs.

  1. Azure Blob Storage
  2. Download missing HDF5 files
  3. Local simulation runs

Azure HDFS Sync illustration

Azure HDFS Sync architecture diagram from the project repository
Public engineering projectGitHub ↗

Problem

Repeated analysis of geophysical simulation outputs requires an organized local copy without re-downloading completed runs.

My role

Built a Python utility for retrieving and organizing simulation outputs.

Approach

Results

Stack

Next projectNASA Intelligence Chat ↗