Dataset · ccdv
arXiv: scientific article summarization
ccdv/arxiv-summarization
arXiv papers and abstracts for training long-document summarization in English.
What's inside
A dataset for long-document summarization adapted from a separate repository. It contains papers and abstracts; the source data was pre-tokenized, so the text is returned concatenated with paragraph line breaks. Compatible with run_summarization.py from Transformers when adding the ("article", "abstract") mapping. Language is en, size 100k-1M.
What you can do with it
- Training scientific article summarization
- Fine-tuning LLMs or LoRA for abstracts
- Evaluating models on long documents
- Tasks
- Summarization, Text generation
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 8K
- Likes
- 141
Models trained on it
- chopratejas/kompress-baseToken classification 711
- farleyknight-org-username/arxiv-summarization-t5-small 104
- JustinDu/BARTxivSummarization 94
- UNIST-Eunchan/Research-Paper-Summarization-Pegasus-x-ArXivSummarization 42
- gabe-zhang/Llama-PaperSummarization-LoRASummarization 30
- adityashukzy/bart-base-arxiv-sum-session-1Summarization 23
- LookUpMark/sigext-arxiv-en-allenai-1k-080tToken classification 19
- LookUpMark/sigext-arxiv-en-xlmr-1k-070tToken classification 17
How to load it
from datasets import load_dataset
ds = load_dataset("ccdv/arxiv-summarization")