Skip to content

Dataset · bakrianoo

Corpus for small LLM: Egypt and Wikipedia

bakrianoo/jabarti-llm-dataset

Clean sectioned corpus in Arabic and English for pre-training a small LLM.

What's inside

The dataset combines a curated collection on Egyptian history and general Wikipedia coverage from CohereLabs/wikipedia-2023-11-embed-multilingual-v3. Each pre-training record is a continuous fragment of 120-1500 characters without article or section titles. The origin is indicated in the ds_source field. Languages are Arabic and English, modalities are tabular data and text, and size ranges from 1M to 10M records.

What you can do with it

  • Pre-training a small Arabic-English LLM
  • LoRA fine-tuning for text generation
  • Evaluating models on Egyptian history questions
Tasks
Text generation, Question answering
Languages
Arabic, English
Size
1M–10M rows
Downloads this month
4.8K
Likes
164

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("bakrianoo/jabarti-llm-dataset")