Skip to content

Dataset · E4DRR

ECMWF weather forecast virtual parquet indexes

E4DRR/gik-ecmwf-par

Lightweight parquet files with links to ECMWF IFS weather forecasts on AWS S3 for analysis without downloading gigabytes.

What's inside

The dataset contains less than a thousand compact parquet files, each around 140 KB in size. Instead of actual weather data, they contain tables with byte offsets and links to the original ECMWF IFS ensemble forecast GRIB files on AWS S3. This approach enables parallel analysis using Dask without downloading 3 to 4 GB files.

What you can do with it

  • Fast cloud access to ECMWF weather forecasts for training meteorological models
  • Building scalable climate data processing pipelines via Dask
Size
up to 1K rows
Downloads this month
43K
Likes
4

How to load it

from datasets import load_dataset

ds = load_dataset("E4DRR/gik-ecmwf-par")