Dataset · permutans
Metadata for 2.5M arXiv papers by subject and date
permutans/arxiv-papers-by-subject
A reorganized corpus of English arXiv scientific paper metadata broken down by subject, year, and month for selective downloading.
What's inside
The dataset collects text metadata for over 2.5 million English scientific publications from the arXiv portal. It is a reworked version of another repository where files are organized in a folder hierarchy by discipline codes, years, and months, enabling users to download only the necessary slices without downloading the entire archive.
What you can do with it
- Train text generation and feature extraction models in narrow scientific domains
- Gather thematic publication samples for specific periods for analytics and RAG
- Tasks
- Text generation, Embeddings
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 102.6K
- Likes
- 39
How to load it
from datasets import load_dataset
ds = load_dataset("permutans/arxiv-papers-by-subject")