Skip to content

Dataset · AdaptLLM

Medical benchmark for evaluating fine-tuned LLMs

AdaptLLM/medicine-tasks

Collection of medical classification and question-answering tasks from an ICLR 2024 paper to test model knowledge.

What's inside

Contains 1 to 10 thousand examples in English in text and tabular formats. The dataset was prepared by the authors of the ICLR 2024 domain adaptation study for language models and is intended to evaluate their ability to answer questions and classify texts.

What you can do with it

  • Evaluating the quality of medical LLMs in question-answering tasks
  • Testing models on zero-shot classification of medical texts
  • Verifying the retention of instruction-following skills after domain fine-tuning
Tasks
Text classification, Question answering, Zero shot classification
Languages
English
Size
1K–10K rows
Downloads this month
381
Likes
33

How to load it

from datasets import load_dataset

ds = load_dataset("AdaptLLM/medicine-tasks")