Dataset · OpenLLM-France
Multilingual text corpus for pre-training Lucie-7B LLM
OpenLLM-France/Lucie-Training-Dataset
Tens of billions of text examples in five European languages and code for pre-training base language models.
What's inside
The corpus contains 10 to 100 billion records in English, French, German, Spanish, and Italian, as well as examples in various programming languages. The data is gathered from web pages, video subtitles, research articles, digital books, newspapers, and magazines, some of which were processed via OCR. The dataset was used to pre-train the Lucie-7B base model.
What you can do with it
- Pre-training multilingual base LLMs
- Fine-tuning models to improve knowledge of European languages and code
- Researching filtering and balancing of heterogeneous text corpora
- Tasks
- Text generation
- Languages
- English, French, German, Spanish, Italian, Code
- Size
- 10B–100B rows
- Downloads this month
- 12.9K
- Likes
- 40
Models trained on it
- OpenLLM-France/Lucie-7BText generation 745
- tensorblock/OpenLLM-France_Lucie-7B-GGUFText generation 68
- alexgusevski/Lucie-7B-mlxText generation 21
- alexgusevski/Lucie-7B-q3-mlxText generation 19
- alexgusevski/Lucie-7B-q6-mlxText generation 18
- alexgusevski/Lucie-7B-q8-mlxText generation 15
- alexgusevski/Lucie-7B-q4-mlxText generation 14
- kerweros/Lucie-7BText generation 13
How to load it
from datasets import load_dataset
ds = load_dataset("OpenLLM-France/Lucie-Training-Dataset")