Dataset · unimelb-nlp
Multilingual NER on Wikipedia articles
unimelb-nlp/wikiann
176 languages, LOC/PER/ORG markup in IOB2 based on Wikipedia articles.
What's inside
WikiANN (PAN-X) is a multilingual dataset for NER. Wikipedia articles are tagged with LOC, PER, and ORG in the IOB2 format. The version corresponds to the balanced train/dev/test splits of Rahimi et al. (2019) and covers 176 out of 282 languages of the original corpus. Size ranges from 1M to 10M examples.
What you can do with it
- Training a multilingual NER model
- Testing token-classification on low-resource languages
- Fine-tuning XLM-R or mBERT for entity extraction
- Tasks
- Token classification
- Languages
- Acehnese, Afrikaans, Albanian, Amharic, Aragonese, Old English, Arabic, Aramaic and 4 more
- Size
- 1M–10M rows
- Downloads this month
- 15.7K
- Likes
- 126
Models trained on it
- emi08/maske-berturk-baselineToken classification 146
- emi08/maske-modernbert-tr-bakeoffToken classification 75
- temsa/OpenMed-PPSN-v5_1Token classification 49
- Elafnawaf/gliner2-arabic-saudiToken classification 49
- burakozmen/vbm695-clinical-turkish-nerToken classification 49
- Tevfik-istanbullu/camelbert-ner-arabicToken classification 46
- temsa/OpenMed-mLiteClinical-IrishPPSN-135M-v1Token classification 44
- Elafnawaf/gliner2-arabic-multitaskToken classification 42
How to load it
from datasets import load_dataset
ds = load_dataset("unimelb-nlp/wikiann")