Dataset · m-a-p
First corpus for pretraining music LLMs
m-a-p/MusicPile
English corpus of 5.17 million texts (4.16 billion tokens) about music, featuring ABC notation sheet music and books.
What's inside
A large English text dataset containing 5.17 million examples and about 4.16 billion tokens. It includes web texts, encyclopedias, music books, YouTube music video descriptions, musical pieces in ABC notation, as well as math and code. The data is prepared for the pretraining stage and is easily loaded via the datasets library.
What you can do with it
- Pretraining and fine-tuning LLMs for understanding music theory and generating sheet music in ABC format
- Creating specialized music assistants and chatbots
- Tasks
- Text generation
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 715
- Likes
- 60
How to load it
from datasets import load_dataset
ds = load_dataset("m-a-p/MusicPile")