Dataset · TucanoBR
780 GB of Portuguese language texts
TucanoBR/GigaVerbo
A summarized text corpus of 200 billion tokens for training Portuguese language models.
What's inside
A large-scale dataset of 780 GB (over 200 billion tokens) in Portuguese, compiled by combining several open datasets from Hugging Face. It is presented in text and tabular formats.
What you can do with it
- Pretraining large language models in Portuguese
- Language adaptation of existing multilingual LLMs for Portuguese
- Tasks
- Text generation
- Languages
- Portuguese
- Size
- 100M–1B rows
- Downloads this month
- 598
- Likes
- 42
Models trained on it
- TucanoBR/Tucano-160mText generation 2.7K
- AIInstituteLNCC/manaca-1b-baseText generation 1K
- TucanoBR/Tucano-2b4Text generation 960
- TucanoBR/Tucano-630mText generation 900
- TucanoBR/Tucano-1b1Text generation 663
- mradermacher/manaca-1b-base-i1-GGUFText generation 631
- mradermacher/Prosodia_T1.7B-Instruct-GGUF 326
- mradermacher/manaca-1b-base-GGUFText generation 232
How to load it
from datasets import load_dataset
ds = load_dataset("TucanoBR/GigaVerbo")