Dataset · lucasmccabe
Logic tasks from China civil service exam (in English)
lucasmccabe/logiqa
English collection of logical reasoning and critical analysis tasks for evaluating model reasoning abilities.
What's inside
The set consists of logical comprehension tasks taken from open questions of the National Civil Service Examination of China. The repository features between 1 and 10 thousand question-answering examples in English only (the original Chinese version is available on the project website). The questions were originally designed to test critical thinking and problem-solving skills.
What you can do with it
- Benchmarking logical inference and reasoning in LLMs
- SFT of models for critical thinking tasks
- Evaluating response quality in multiple-choice or question-answering format
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 4.3K
- Likes
- 42
Models trained on it
- tasksource/deberta-small-long-nliZero shot classification 9.3K
- sileod/deberta-v3-base-tasksource-nliZero shot classification 8K
- sileod/deberta-v3-large-tasksource-nliZero shot classification 1.8K
- tasksource/deberta-base-long-nliZero shot classification 1.4K
- sileod/deberta-v3-xsmall-tasksource-nliZero shot classification 240
- rudranshjoshi/circuit 96
- saytes/SoT_DistilBERTText classification 45
- ITLL/deberta-v3-base-tasksource-nliZero shot classification 41
How to load it
from datasets import load_dataset
ds = load_dataset("lucasmccabe/logiqa")