Skip to content

Dataset · open-index

Complete Open Library book catalog in parquet

open-index/open-library

Over 150 million bibliographic records from the Open Library project for NLP, search, and data extraction tasks.

What's inside

Contains a complete snapshot of the Internet Archive's Open Library database in parquet format. The dataset contains over 150 million records across 11 entity types: from ISBN codes and author biographies to reading lists and Wikidata links. The data was collected by the community in English and other languages.

What you can do with it

  • Training named entity recognition (NER) and text classification systems using bibliographic data
  • Building search engines, recommendation services, and book-based RAG pipelines
  • Entity linking using Wikidata links
Tasks
Text generation, Embeddings, Text classification, Question answering
Languages
English, Multiple languages
Size
100M–1B rows
Downloads this month
2.1K
Likes
11

How to load it

from datasets import load_dataset

ds = load_dataset("open-index/open-library")