Skip to content

Dataset · rmyeid

Multilingual corpus for named entity recognition

rmyeid/polyglot_ner

Automatically annotated texts in 40 languages for training NER models.

What's inside

Contains training texts from Wikipedia with automatic named entity annotation and coreference resolution. The data is gathered from Wikipedia and Freebase for 40 languages (Arabic, Bulgarian, Catalan, Czech, Danish, and German are noted in the metadata). Each language is separated into its own config.

What you can do with it

  • Training multilingual models for the NER task (token-classification)
  • Fine-tuning models for entity extraction in low-resource and European languages
Tasks
Token classification
Languages
Arabic, Bulgarian, Catalan, Czech, Danish, German, Greek, English and 4 more
Downloads this month
723
Likes
40

How to load it

from datasets import load_dataset

ds = load_dataset("rmyeid/polyglot_ner")