Dataset · Muennighoff
XWinograd multilingual coreference resolution test
Muennighoff/xwinograd
Winograd schemas in six languages, including Russian, to evaluate LLM language understanding.
What's inside
A multilingual version of the Winograd Schema Challenge used in cross-lingual fine-tuning research. It contains text examples in English (2,325), Japanese (959), Chinese (504, including examples from cluewsc2020), Russian (315), Portuguese (263), and French (83).
What you can do with it
- Testing the capability of multilingual LLMs to understand context and resolve pronoun ambiguity
- Evaluating cross-lingual generalization skills after SFT
- Languages
- English, French, Japanese, Portuguese, Russian, Chinese
- Size
- 1K–10K rows
- Downloads this month
- 39.3K
- Likes
- 18
How to load it
from datasets import load_dataset
ds = load_dataset("Muennighoff/xwinograd")