Dataset · MoritzLaurer
Multilingual NLI corpus across 26 languages (2.73M pairs)
MoritzLaurer/multilingual-NLI-26lang-2mil7
Massive set of 2.73 million text pairs in 26 languages for training logical inference and zero-shot classification models.
What's inside
The dataset contains 2,730,000 text pairs for the Natural Language Inference (NLI) task across 26 languages (including Chinese, Japanese, Arabic, Korean, German, and others). Created using open machine learning translation models based on well-known English corpora MultiNLI, Fever-NLI, ANLI, LingNLI, and WANLI.
What you can do with it
- Training multilingual models for Natural Language Inference (NLI)
- Training text classifiers in zero-shot mode for 26 languages
- Evaluating cross-lingual natural language understanding quality
- Tasks
- Text classification
- Languages
- Multilingual, Chinese, Japanese, Arabic, Korean, German, French, Spanish and 4 more
- Size
- 1M–10M rows
- Downloads this month
- 1.7K
- Likes
- 52
Models trained on it
- MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7Zero shot classification 714.4K
- pkshatech/GLuCoSE-base-jaSentence similarity 31.8K
- alexandrainst/scandi-nli-largeZero shot classification 3.3K
- onnx-community/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7-ONNXZero shot classification 1.1K
- sileod/mdeberta-v3-base-tasksource-nliZero shot classification 262
- BalaRajesh1/mmbert-small-nliText classification 189
- ilos-vigil/bigbird-small-indonesian-nliZero shot classification 182
- iara-project/e5-large-matryoshka-sts-ptSentence similarity 124
How to load it
from datasets import load_dataset
ds = load_dataset("MoritzLaurer/multilingual-NLI-26lang-2mil7")