Skip to content

Dataset · saheedniyi

Four-language Nigerian web text corpus

saheedniyi/naijaweb

Over 270k web documents (~230M tokens) in English, Yoruba, Hausa, and Igbo.

What's inside

A text corpus in parquet format containing over 270k documents (about 230M GPT-2 tokens) gathered from popular Nigerian web pages. Materials are presented in English, Yoruba, Hausa, and Igbo, accompanied by metadata on language, domain, token count, and text quality score. The dataset was created similarly to FineWeb to convey Nigerian linguistic and cultural context.

What you can do with it

  • Pre-training and language adaptation of LLMs for low-resource African languages
  • Filtering high-quality educational texts using the built-in int_score metric
Tasks
Text generation
Languages
English, Yoruba, Hausa, Igbo
Size
100K–1M rows
Downloads this month
70
Likes
39

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("saheedniyi/naijaweb")