Dataset · amazon-agi
50 million instructions for speech language models from Amazon
amazon-agi/SIFT-50M
Large-scale multilingual corpus based on 14,000 hours of audio for training speech understanding and generation.
What's inside
SIFT-50M includes 50 million examples for pre-training and SFT fine-tuning of speech language models in five languages: English, German, French, Italian, and Spanish. The corpus was compiled by Amazon AGI based on open speech databases with a total duration of 14,000 hours using LLMs and expert models. The data covers speech understanding, audio classification, and controlled voice synthesis tasks.
What you can do with it
- Pre-training and SFT fine-tuning of multimodal audio-text LLMs
- Training controlled speech synthesis systems (text-to-speech)
- Creating multilingual voice assistants
- Tasks
- Audio text → text, Audio classification, Text → speech, Audio → audio
- Languages
- English, German, French, Italian, Spanish
- Size
- 10M–100M rows
- Downloads this month
- 4.3K
- Likes
- 39
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("amazon-agi/SIFT-50M")