Skip to content

Dataset · Qdrant

10B-embedding vector search benchmark

Qdrant/FineWeb-10B

10 billion dense and sparse vectors based on FineWeb with ground truth references for evaluating search systems.

What's inside

The Qdrant-FineWeb-10B (Q-FineWeb-10B) benchmark includes 10 billion dense and 10 billion sparse embeddings generated by the Alibaba-NLP/gte-multilingual-base model using English FineWeb texts. It also features source texts with metadata and exact ground truth search results for ~120,000 MS MARCO queries.

What you can do with it

  • Load testing and benchmarking vector databases
  • Evaluating hybrid (dense plus sparse) search quality at ultra-large scales
  • Researching approximate nearest neighbor (ANN) search algorithms
Languages
English
Size
10B–100B rows
Downloads this month
61.2K
Likes
28

How to load it

from datasets import load_dataset

ds = load_dataset("Qdrant/FineWeb-10B")