Dataset · Qdrant
10B-embedding vector search benchmark
Qdrant/FineWeb-10B
10 billion dense and sparse vectors based on FineWeb with ground truth references for evaluating search systems.
What's inside
The Qdrant-FineWeb-10B (Q-FineWeb-10B) benchmark includes 10 billion dense and 10 billion sparse embeddings generated by the Alibaba-NLP/gte-multilingual-base model using English FineWeb texts. It also features source texts with metadata and exact ground truth search results for ~120,000 MS MARCO queries.
What you can do with it
- Load testing and benchmarking vector databases
- Evaluating hybrid (dense plus sparse) search quality at ultra-large scales
- Researching approximate nearest neighbor (ANN) search algorithms
- Languages
- English
- Size
- 10B–100B rows
- Downloads this month
- 61.2K
- Likes
- 28
How to load it
from datasets import load_dataset
ds = load_dataset("Qdrant/FineWeb-10B")