# Analysis of the Cambridge Multiple-Choice Questions Reading Dataset with a Focus on Candidate Response Distribution

arXiv: https://arxiv.org/abs/2306.13047 · HTML: https://mlsanity.com/paper/2306.13047/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 22 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://mlsanity.com/paper/1909.00277/) (2019)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension](https://mlsanity.com/paper/1902.00164/) (2019)
- [Multiple-Choice Question Generation: Towards an Automated Assessment Framework](https://mlsanity.com/paper/2209.11830/) (2022)
- [World Knowledge in Multiple Choice Reading Comprehension](https://mlsanity.com/paper/2211.07040/) (2022)

## Built on by (most cited first, 7 of 7)

- [Do LLMs Give Psychometrically Plausible Responses in Educational Assessments?](https://mlsanity.com/paper/2506.09796/) (2025)
- [Question Difficulty Ranking for Multiple-Choice Reading Comprehension](https://mlsanity.com/paper/2404.10704/) (2024)
- [Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items?](https://mlsanity.com/paper/2510.25064/) (2025)
- [Assessing Distractors in Multiple-Choice Tests](https://mlsanity.com/paper/2311.04554/) (2023)
- [LLMs Struggle to Measure What Distinguishes Students of Different Proficiency Levels: A Study of Item Discrimination in Reading Comprehension Assessment](https://mlsanity.com/paper/2606.18709/) (2026)
- [Finetuning LLMs for Comparative Assessment Tasks](https://mlsanity.com/paper/2409.15979/) (2024)
- [An Information-Theoretic Approach to Analyze NLP Classification Tasks](https://mlsanity.com/paper/2402.00978/) (2024)

## Abstract (arXiv)

Multiple choice exams are widely used to assess candidates across a diverse range of domains and tasks. To moderate question quality, newly proposed questions often pass through pre-test evaluation stages before being deployed into real-world exams. Currently, this evaluation process is manually intensive, which can lead to time lags in the question development cycle. Streamlining this process via automation can significantly enhance efficiency, however, there's a current lack of datasets with adequate pre-test analysis information. In this paper we analyse a subset of the public Cambridge Multiple-Choice Questions Reading Database released by Cambridge University Press & Assessment; a multiple-choice comprehension dataset of questions at different target levels, with corresponding candidate selection distributions. We introduce the task of candidate distribution matching, propose several evaluation metrics for the task, and demonstrate that automatic systems trained on RACE++ can be leveraged as baselines for our task. We further demonstrate that these automatic systems can be used for practical pre-test evaluation tasks such as detecting underperforming distractors, where our detection systems can automatically identify poor distractors that few candidates select.