# Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration

arXiv: https://arxiv.org/abs/2406.13578 · HTML: https://mlsanity.com/paper/2406.13578/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering](https://mlsanity.com/paper/2104.06378/) (2021)
- [Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings](https://mlsanity.com/paper/2002.05969/) (2020)
- [GreaseLM: Graph REASoning Enhanced Language Models for Question Answering](https://mlsanity.com/paper/2201.08860/) (2022)
- [Deep Bidirectional Language-Knowledge Graph Pretraining](https://mlsanity.com/paper/2210.09338/) (2022)
- [Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering](https://mlsanity.com/paper/2005.00646/) (2020)
- [Knowledge-Driven Distractor Generation for Cloze-style Multiple Choice Questions](https://mlsanity.com/paper/2004.09853/) (2020)

## Built on by (most cited first, 11 of 11)

- [Answer Matching Outperforms Multiple Choice for Language Model Evaluation](https://mlsanity.com/paper/2507.02856/) (2025)
- [Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation](https://mlsanity.com/paper/2501.03225/) (2025)
- [Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction](https://mlsanity.com/paper/2501.13125/) (2025)
- [LookAlike: Consistent Distractor Generation in Math MCQs](https://mlsanity.com/paper/2505.01903/) (2025)
- [Wrong Answers Can Also Be Useful: PlausibleQA -- A Large-Scale QA Dataset with Answer Plausibility Scores](https://mlsanity.com/paper/2502.16358/) (2025)
- [LLM Distillation for Efficient Few-Shot Multiple Choice Question Answering](https://mlsanity.com/paper/2412.09807/) (2024)
- [AutoDrive-QA: A Multiple-Choice Benchmark for Vision-Language Evaluation in Urban Autonomous Driving](https://mlsanity.com/paper/2503.15778/) (2025)
- [Cognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models](https://mlsanity.com/paper/2602.03704/) (2026)
- [D-GEN: Automatic Distractor Generation and Evaluation for Reliable Assessment of Generative Model](https://mlsanity.com/paper/2504.13439/) (2025)
- [Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT](https://mlsanity.com/paper/2511.17405/) (2025)
- [DualReward: A Dynamic Reinforcement Learning Framework for Cloze Tests Distractor Generation](https://mlsanity.com/paper/2507.11875/) (2025)

## Abstract (arXiv)

In this paper, we tackle the task of distractor generation (DG) for multiple-choice questions. Our study introduces two key designs. First, we propose textitretrieval augmented pretraining, which involves refining the language model pretraining to align it more closely with the downstream task of DG. Second, we explore the integration of knowledge graphs to enhance the performance of DG. Through experiments with benchmarking datasets, we show that our models significantly outperform the state-of-the-art results. Our best-performing model advances the F1@3 score from 14.80 to 16.47 in MCQ dataset and from 15.92 to 16.50 in Sciq dataset.