Skip to content

Dataset · HuggingFaceFW

3 Trillion Token PDF Corpus

HuggingFaceFW/finepdfs

The largest open text corpus from PDFs: 3 trillion tokens, 475 million documents, and 1,733 languages.

What's inside

FinePDFs is a public corpus collected exclusively from PDF documents. It contains approximately 3 trillion tokens, 475 million documents, and texts in 1,733 languages; the metadata also specifies tabular and text fields. The authors describe it as the largest open PDF corpus created due to the difficulty and cost of text extraction.

What you can do with it

  • Pre-training multilingual LLMs on documents
  • Fine-tuning models on PDF texts
  • Testing tokenizers on multilingual documents
Tasks
Text generation
Languages
aai, aak, aau, aaz, aba, abi, Abkhazian, abn and 4 more
Size
100M–1B rows
Downloads this month
85.5K
Likes
962

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceFW/finepdfs")