Dataset · globis-university
Cleaned Japanese text corpus from Aozora Bunko library
globis-university/aozorabunko-clean
Public domain Japanese fiction and nonfiction literature corpus.
What's inside
The dataset contains between 10,000 and 100,000 cleaned Japanese texts collected from the Aozora Bunko online library, which publishes public domain works. The authors prepared the data in a machine learning-friendly format and published the parsing code on GitHub. The dataset is intended for text generation and classification tasks.
What you can do with it
- Fine-tuning language models or LoRA on classic and literary Japanese language
- Training text classification and Japanese literature style analysis models
- Tasks
- Text generation, Text classification
- Languages
- Japanese
- Size
- 10K–100K rows
- Downloads this month
- 1.5K
- Likes
- 49
Models trained on it
- nvidia/NVIDIA-Nemotron-Nano-9B-v2-JapaneseText generation 3.2K
- mradermacher/NVIDIA-Nemotron-Nano-9B-v2-Japanese-i1-GGUF 2.7K
- JustANormalTinkerer/hayai-ocr-v2Image → text 2.4K
- JustANormalTinkerer/hayai-ocr-v2.5-novaImage → text 1.8K
- mradermacher/NVIDIA-Nemotron-Nano-9B-v2-Japanese-GGUF 1.1K
- globis-university/deberta-v3-japanese-baseToken classification 709
- akineAItech/Noberai-7B-V001-2048-alphaText generation 372
- KoichiYasuoka/modernbert-base-japanese-charFill mask 300
How to load it
from datasets import load_dataset
ds = load_dataset("globis-university/aozorabunko-clean")