# Making Neural QA as Simple as Possible but not Simpler

arXiv: https://arxiv.org/abs/1703.04816 · HTML: https://mlsanity.com/paper/1703.04816/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2017.

## Built on by (most cited first, 30 of 33)

- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [DataPerf: Benchmarks for Data-Centric AI Development](https://mlsanity.com/paper/2207.10062/) (2022)
- [Biomedical Question Answering: A Survey of Approaches and Challenges](https://mlsanity.com/paper/2102.05281/) (2021)
- [Why Machine Reading Comprehension Models Learn Shortcuts?](https://mlsanity.com/paper/2106.01024/) (2021)
- [CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation](https://mlsanity.com/paper/2211.00295/) (2022)
- [Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants](https://mlsanity.com/paper/2112.09062/) (2021)
- [Self-Supervised Test-Time Learning for Reading Comprehension](https://mlsanity.com/paper/2103.11263/) (2021)
- [Do Question Answering Modeling Improvements Hold Across Benchmarks?](https://mlsanity.com/paper/2102.01065/) (2021)
- [Sequence tagging for biomedical extractive question answering](https://mlsanity.com/paper/2104.07535/) (2021)
- [CodeQueries: A Dataset of Semantic Queries over Code](https://mlsanity.com/paper/2209.08372/) (2022)
- [DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries](https://mlsanity.com/paper/2310.04678/) (2023)
- [Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models](https://mlsanity.com/paper/2402.02099/) (2024)
- [Which Shortcut Solution Do Question Answering Models Prefer to Learn?](https://mlsanity.com/paper/2211.16220/) (2022)
- [What Has Been Lost with Synthetic Evaluation?](https://mlsanity.com/paper/2505.22830/) (2025)
- [MultiWikiQA: A Reading Comprehension Benchmark in 300+ Languages](https://mlsanity.com/paper/2509.04111/) (2025)
- [Look to the Right: Mitigating Relative Position Bias in Extractive Question Answering](https://mlsanity.com/paper/2210.14541/) (2022)
- [Learning to Generalize for Cross-domain QA](https://mlsanity.com/paper/2305.08208/) (2023)
- [TIGQA:An Expert Annotated Question Answering Dataset in Tigrinya](https://mlsanity.com/paper/2404.17194/) (2024)
- [Pir'a: A Bilingual Portuguese-English Dataset for Question-Answering about the Ocean](https://mlsanity.com/paper/2202.02398/) (2022)
- [Design and Development of Rule-based open-domain Question-Answering System on SQuAD v2.0 Dataset](https://mlsanity.com/paper/2204.09659/) (2022)
- [A Coarse to Fine Question Answering System based on Reinforcement Learning](https://mlsanity.com/paper/2106.00257/) (2021)
- [Interactive Machine Comprehension with Dynamic Knowledge Graphs](https://mlsanity.com/paper/2109.00077/) (2021)
- [To Answer or Not to Answer? Improving Machine Reading Comprehension Model with Span-based Contrastive Learning](https://mlsanity.com/paper/2208.01299/) (2022)
- [Long-context Non-factoid Question Answering in Indic Languages](https://mlsanity.com/paper/2504.13615/) (2025)
- [Medical Knowledge Graph QA for Drug-Drug Interaction Prediction based on Multi-hop Machine Reading Comprehension](https://mlsanity.com/paper/2212.09400/) (2022)
- [Self-Supervised Position Debiasing for Large Language Models](https://mlsanity.com/paper/2401.01218/) (2024)
- [How the Advent of Ubiquitous Large Language Models both Stymie and Turbocharge Dynamic Adversarial Question Generation](https://mlsanity.com/paper/2401.11185/) (2024)
- [Transformer-based Language Models for Factoid Question Answering at BioASQ9b](https://mlsanity.com/paper/2109.07185/) (2021)
- [An Automated Question-Answering Framework Based on Evolution Algorithm](https://mlsanity.com/paper/2201.10797/) (2022)
- [MRCLens: an MRC Dataset Bias Detection Toolkit](https://mlsanity.com/paper/2207.08943/) (2022)

## Abstract (arXiv)

Recent development of large-scale question answering (QA) datasets triggered a substantial amount of research into end-to-end neural architectures for QA. Increasingly complex systems have been conceived without comparison to simpler neural baseline systems that would justify their complexity. In this work, we propose a simple heuristic that guides the development of neural baseline systems for the extractive QA task. We find that there are two ingredients necessary for building a high-performing neural QA system: first, the awareness of question words while processing the context and second, a composition function that goes beyond simple bag-of-words modeling, such as recurrent neural networks. Our results show that FastQA, a system that meets these two requirements, can achieve very competitive performance compared with existing models. We argue that this surprising finding puts results of previous systems and the complexity of recent QA datasets into perspective.