Dataset · HuggingFaceFW
3 Trillion Token PDF Corpus
HuggingFaceFW/finepdfs
The largest open text corpus from PDFs: 3 trillion tokens, 475 million documents, and 1,733 languages.
What's inside
FinePDFs is a public corpus collected exclusively from PDF documents. It contains approximately 3 trillion tokens, 475 million documents, and texts in 1,733 languages; the metadata also specifies tabular and text fields. The authors describe it as the largest open PDF corpus created due to the difficulty and cost of text extraction.
What you can do with it
- Pre-training multilingual LLMs on documents
- Fine-tuning models on PDF texts
- Testing tokenizers on multilingual documents
- Tasks
- Text generation
- Languages
- aai, aak, aau, aaz, aba, abi, Abkhazian, abn and 4 more
- Size
- 100M–1B rows
- Downloads this month
- 85.5K
- Likes
- 962
Models trained on it
- permitt/galton-modernbertic-largeFill mask 27.7K
- neuralcrew/neutrino-instructText generation 14.4K
- mradermacher/Apex-2-Base-i1-GGUF 1.4K
- HPLT/NorOLMo-13BText generation 1.2K
- HPLT/FinOLMo-13BText generation 811
- TildeAI/TildeOpen-30b-64kText generation 559
- mradermacher/Villanova-2B-Base-2603-i1-GGUF 465
- mradermacher/Apex-2-Base-GGUF 444
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/finepdfs")