Dataset · RussianNLP
Russian SuperGLUE Benchmark for Russian Language
RussianNLP/russian_super_glue
A set of nine challenging language understanding and logic tasks for evaluating Russian-language models.
What's inside
This is a Russian-language counterpart to the well-known SuperGLUE benchmark, created from scratch to diagnose general intellectual skills of language models. It contains between 100 thousand and 1 million examples for text classification, question answering, zero-shot classification, and generation tasks that test logical reasoning and common sense. Baselines, human-level evaluations, and an open testing framework are attached to the dataset.
What you can do with it
- Comprehensive evaluation of Russian-language LLM quality on text understanding and logic
- Comparing models with baselines and human results in a unified pipeline
- Tasks
- Text classification, Question answering, Zero shot classification, Text generation
- Languages
- Russian
- Size
- 100K–1M rows
- Downloads this month
- 1.1K
- Likes
- 36
Models trained on it
- deepvk/USER-bge-m3Sentence similarity 403.5K
- deepvk/USER-baseSentence similarity 29.4K
- 6E6E/ru-decision-4bText classification 113
- alela32/USER-bge-m3-Q8_0-GGUFSentence similarity 111
- lucky-striker/gemma4-e2b-decision-ruText classification 67
- Content-AI/USER-bge-m3-Q8_0-GGUFSentence similarity 59
- sparkhonyuk/USER-bge-m3-Q5_K_M-GGUFSentence similarity 58
- cm4ker/USER-bge-m3-Q4_K_M-GGUFSentence similarity 54
How to load it
from datasets import load_dataset
ds = load_dataset("RussianNLP/russian_super_glue")