Skip to content

Dataset · Muennighoff

XWinograd multilingual coreference resolution test

Muennighoff/xwinograd

Winograd schemas in six languages, including Russian, to evaluate LLM language understanding.

What's inside

A multilingual version of the Winograd Schema Challenge used in cross-lingual fine-tuning research. It contains text examples in English (2,325), Japanese (959), Chinese (504, including examples from cluewsc2020), Russian (315), Portuguese (263), and French (83).

What you can do with it

  • Testing the capability of multilingual LLMs to understand context and resolve pronoun ambiguity
  • Evaluating cross-lingual generalization skills after SFT
Languages
English, French, Japanese, Portuguese, Russian, Chinese
Size
1K–10K rows
Downloads this month
39.3K
Likes
18

How to load it

from datasets import load_dataset

ds = load_dataset("Muennighoff/xwinograd")