Skip to content

Dataset · mteb

Multilingual news similarity benchmark (STS22.v2)

mteb/sts22-crosslingual-sts

Cleaned set of multilingual news article pairs for evaluating text embedding models.

What's inside

The dataset contains news text pairs with semantic similarity scores from SemEval 2022 Task 8, part of the MTEB benchmark suite. The volume ranges from 10 to 100 thousand records in Arabic, Chinese, German, English, French, Italian, and other languages. In the second version, pairs with one empty sentence were removed.

What you can do with it

  • Evaluating and comparing multilingual embedding models (bi-encoders)
  • Calibrating semantic search thresholds and news feed deduplication
Tasks
Sentence similarity
Languages
Arabic, Chinese, German, English, French, Italian, Polish, Russian and 2 more
Size
10K–100K rows
Downloads this month
21.8K
Likes
16

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("mteb/sts22-crosslingual-sts")