Skip to content

Dataset · RussianNLP

Russian SuperGLUE Benchmark for Russian Language

RussianNLP/russian_super_glue

A set of nine challenging language understanding and logic tasks for evaluating Russian-language models.

What's inside

This is a Russian-language counterpart to the well-known SuperGLUE benchmark, created from scratch to diagnose general intellectual skills of language models. It contains between 100 thousand and 1 million examples for text classification, question answering, zero-shot classification, and generation tasks that test logical reasoning and common sense. Baselines, human-level evaluations, and an open testing framework are attached to the dataset.

What you can do with it

  • Comprehensive evaluation of Russian-language LLM quality on text understanding and logic
  • Comparing models with baselines and human results in a unified pipeline
Tasks
Text classification, Question answering, Zero shot classification, Text generation
Languages
Russian
Size
100K–1M rows
Downloads this month
1.1K
Likes
36

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("RussianNLP/russian_super_glue")