Dataset · juletxara
MGSM multilingual grade-school math benchmark
juletxara/mgsm
250 math problems from GSM8K translated by humans into 10 languages, including Russian, to evaluate reasoning.
What's inside
The benchmark includes 250 elementary school-level word math problems taken from the GSM8K set. Each problem was translated by professional annotators into 10 languages, including Russian, English, Spanish, German, French, Chinese, and Japanese. The set is designed to test models' chain-of-thought reasoning capabilities across different languages.
What you can do with it
- Evaluating LLM mathematical capabilities and chain-of-thought reasoning quality in Russian and other languages
- Comparing cross-lingual transfer of logical inference skills in multilingual models
- Tasks
- Text generation
- Languages
- English, Spanish, French, German, Russian, Chinese, Japanese, Thai and 2 more
- Size
- 1K–10K rows
- Downloads this month
- 29.7K
- Likes
- 45
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("juletxara/mgsm")