Dataset · bakrianoo
Corpus for small LLM: Egypt and Wikipedia
bakrianoo/jabarti-llm-dataset
Clean sectioned corpus in Arabic and English for pre-training a small LLM.
What's inside
The dataset combines a curated collection on Egyptian history and general Wikipedia coverage from CohereLabs/wikipedia-2023-11-embed-multilingual-v3. Each pre-training record is a continuous fragment of 120-1500 characters without article or section titles. The origin is indicated in the ds_source field. Languages are Arabic and English, modalities are tabular data and text, and size ranges from 1M to 10M records.
What you can do with it
- Pre-training a small Arabic-English LLM
- LoRA fine-tuning for text generation
- Evaluating models on Egyptian history questions
- Tasks
- Text generation, Question answering
- Languages
- Arabic, English
- Size
- 1M–10M rows
- Downloads this month
- 4.8K
- Likes
- 164
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("bakrianoo/jabarti-llm-dataset")