Skip to content

Dataset · m-a-p

First corpus for pretraining music LLMs

m-a-p/MusicPile

English corpus of 5.17 million texts (4.16 billion tokens) about music, featuring ABC notation sheet music and books.

What's inside

A large English text dataset containing 5.17 million examples and about 4.16 billion tokens. It includes web texts, encyclopedias, music books, YouTube music video descriptions, musical pieces in ABC notation, as well as math and code. The data is prepared for the pretraining stage and is easily loaded via the datasets library.

What you can do with it

  • Pretraining and fine-tuning LLMs for understanding music theory and generating sheet music in ABC format
  • Creating specialized music assistants and chatbots
Tasks
Text generation
Languages
English
Size
1M–10M rows
Downloads this month
715
Likes
60

How to load it

from datasets import load_dataset

ds = load_dataset("m-a-p/MusicPile")