Dataset · nampdn-ai
Synthetic textbooks for small LLMs
nampdn-ai/tiny-textbooks
100K-1M textbook-style English texts for training small language models.
What's inside
This is a synthetic corpus of textbook-like texts in English. Source texts are selected from falcon-refinedweb and minipile with an emphasis on diversity and quality at a small size. Synthesis was performed on 4x3090 Ti for 500 hours using fine-tuned Nous-Hermes-Llama2-13b. Metadata shows 100K-1M records; modalities are text and tabular data.
What you can do with it
- Pretrain or SFT a small LLM
- Experiments with synthetic training data in the spirit of phi-1.5
- Checking educational text generation quality
- Tasks
- Text generation
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 318
- Likes
- 190
- Access
- on request on Hugging Face
Models trained on it
- NucleusOrg/Nucleus-1B-alpha-1Text generation 274
- igorktech/TinyIGText generation 217
- amusktweewt/tiny-model-700M-chatText generation 147
- SykoSLM/SykoLLM-V6.1Text generation 123
- SykoSLM/SykoLLM-Phi3-TestText generation 81
- tensorblock/Nucleus-1B-alpha-1-GGUF 77
- Josephgflowers/Differential-Attention-Liquid-Metal-Tinyllama 67
- Harley-ml/Tenete-8MText generation 52
How to load it
from datasets import load_dataset
ds = load_dataset("nampdn-ai/tiny-textbooks")