Dataset · jinofy-corp
Multilingual pre-training corpus emphasizing African languages
jinofy-corp/jora_corpus1
A cleaned 18-language text corpus (1B to 10B records) with a focus on low-resource African languages.
What's inside
A cleaned and deduplicated corpus for pre-training JINOVAE Labs' JoraMind language models (JINOVAE's AI division in Africa). Includes web texts, encyclopedias, code, math, medicine, and agriculture across 18 languages—such as Amharic, Hausa, Swahili, Somali, Yoruba, Malagasy, Lingala, Comorian, alongside English, French, Arabic, Spanish, Portuguese, and Hindi. The dataset size spans 1 billion to 10 billion examples.
What you can do with it
- Pre-training or adapting LLMs for African languages
- Fine-tuning multilingual models on subject domains (medicine, agriculture, code)
- Tasks
- Text generation
- Languages
- French, English, Arabic, Spanish, Portuguese, Hindi, Indonesian, Malay and 4 more
- Size
- 1B–10B rows
- Downloads this month
- 25.6K
- Likes
- 6
How to load it
from datasets import load_dataset
ds = load_dataset("jinofy-corp/jora_corpus1")