# Can LLMs Augment Low-Resource Reading Comprehension Datasets? Opportunities and Challenges

arXiv: https://arxiv.org/abs/2309.12426 · HTML: https://mlsanity.com/paper/2309.12426/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022)
- [Generating Datasets with Pretrained Language Models](https://mlsanity.com/paper/2104.07540/) (2021)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Not Enough Data? Deep Learning to the Rescue!](https://mlsanity.com/paper/1911.03118/) (2019)
- [Data Augmentation using Pre-trained Transformer Models](https://mlsanity.com/paper/2003.02245/) (2020)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1809.00732/) (2018)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation](https://mlsanity.com/paper/2104.08678/) (2021)
- [End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems](https://mlsanity.com/paper/2010.06028/) (2020)
- [PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks](https://mlsanity.com/paper/2202.12499/) (2022)
- [Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia](https://mlsanity.com/paper/1805.05942/) (2018)
- [DARE: Data Augmented Relation Extraction with GPT-2](https://mlsanity.com/paper/2004.13845/) (2020)
- [Generate, Annotate, and Learn: NLP with Synthetic Text](https://mlsanity.com/paper/2106.06168/) (2021)
- [Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering](https://mlsanity.com/paper/2010.12643/) (2020)
- [Span Selection Pre-training for Question Answering](https://mlsanity.com/paper/1909.04120/) (2019)
- [BIOMRC: A Dataset for Biomedical Machine Reading Comprehension](https://mlsanity.com/paper/2005.06376/) (2020)
- [PolicyQA: A Reading Comprehension Dataset for Privacy Policies](https://mlsanity.com/paper/2010.02557/) (2020)
- [CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models](https://mlsanity.com/paper/2112.11941/) (2021)
- [The TechQA Dataset](https://mlsanity.com/paper/1911.02984/) (2019)
- [Semi-Supervised QA with Generative Domain-Adaptive Nets](https://mlsanity.com/paper/1702.02206/) (2017)
- [Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions](https://mlsanity.com/paper/2010.10216/) (2020)
- [CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation](https://mlsanity.com/paper/2210.04873/) (2022)
- [Simple and Effective Semi-Supervised Question Answering](https://mlsanity.com/paper/1804.00720/) (2018)

## Built on by (most cited first, 9 of 9)

- [A Survey on Data Augmentation in Large Model Era](https://mlsanity.com/paper/2401.15422/) (2024)
- [What Has Been Lost with Synthetic Evaluation?](https://mlsanity.com/paper/2505.22830/) (2025)
- [LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs](https://mlsanity.com/paper/2408.13467/) (2024)
- [Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2404.17897/) (2024)
- [Position: LLMs Can be Good Tutors in English Education](https://mlsanity.com/paper/2502.05467/) (2025)
- [Measuring Diversity in Synthetic Datasets](https://mlsanity.com/paper/2502.08512/) (2025)
- [Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injection](https://mlsanity.com/paper/2503.17739/) (2025)
- [Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models](https://mlsanity.com/paper/2506.00773/) (2025)
- [Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing](https://mlsanity.com/paper/2603.09205/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive zero shot performance on a wide range of NLP tasks, demonstrating the ability to reason and apply commonsense. A relevant application is to use them for creating high quality synthetic datasets for downstream tasks. In this work, we probe whether GPT-4 can be used to augment existing extractive reading comprehension datasets. Automating data annotation processes has the potential to save large amounts of time, money and effort that goes into manually labelling datasets. In this paper, we evaluate the performance of GPT-4 as a replacement for human annotators for low resource reading comprehension tasks, by comparing performance after fine tuning, and the cost associated with annotation. This work serves to be the first analysis of LLMs as synthetic data augmenters for QA systems, highlighting the unique opportunities and challenges. Additionally, we release augmented versions of low resource datasets, that will allow the research community to create further benchmarks for evaluation of generated datasets.