Dataset · E4DRR
ECMWF weather forecast virtual parquet indexes
E4DRR/gik-ecmwf-par
Lightweight parquet files with links to ECMWF IFS weather forecasts on AWS S3 for analysis without downloading gigabytes.
What's inside
The dataset contains less than a thousand compact parquet files, each around 140 KB in size. Instead of actual weather data, they contain tables with byte offsets and links to the original ECMWF IFS ensemble forecast GRIB files on AWS S3. This approach enables parallel analysis using Dask without downloading 3 to 4 GB files.
What you can do with it
- Fast cloud access to ECMWF weather forecasts for training meteorological models
- Building scalable climate data processing pipelines via Dask
- Size
- up to 1K rows
- Downloads this month
- 43K
- Likes
- 4
How to load it
from datasets import load_dataset
ds = load_dataset("E4DRR/gik-ecmwf-par")