# SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach

arXiv: https://arxiv.org/abs/1804.08798 · HTML: https://mlsanity.com/paper/1804.08798/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2018.

## Built on by (most cited first, 19 of 19)

- [TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph](https://mlsanity.com/paper/2104.07302/) (2021)
- [Complex Knowledge Base Question Answering: A Survey](https://mlsanity.com/paper/2108.06688/) (2021)
- [Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning](https://mlsanity.com/paper/2410.14211/) (2024)
- [An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering](https://mlsanity.com/paper/2303.10368/) (2023)
- [UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text](https://mlsanity.com/paper/2108.08614/) (2021)
- [End-to-End Entity Resolution and Question Answering Using Differentiable Knowledge Graphs](https://mlsanity.com/paper/2109.05817/) (2021)
- [Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformers](https://mlsanity.com/paper/2103.16289/) (2021)
- [Improving Embedded Knowledge Graph Multi-hop Question Answering by introducing Relational Chain Reasoning](https://mlsanity.com/paper/2110.12679/) (2021)
- [Revisiting Backdoor Threat in Federated Instruction Tuning from a Signal Aggregation Perspective](https://mlsanity.com/paper/2602.15671/) (2026)
- [Data Distribution Bottlenecks in Grounding Language Models to Knowledge Bases](https://mlsanity.com/paper/2309.08345/) (2023)
- [Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answering](https://mlsanity.com/paper/2409.15902/) (2024)
- [Two is Better Than One: Answering Complex Questions by Multiple Knowledge Sources with Generalized Links](https://mlsanity.com/paper/2309.05201/) (2023)
- [Knowledge Graph Question Answering via SPARQL Silhouette Generation](https://mlsanity.com/paper/2109.09475/) (2021)
- [Plan Then Retrieve: Reinforcement Learning-Guided Complex Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2510.20691/) (2025)
- [Measuring Linguistic Diversity During COVID-19](https://mlsanity.com/paper/2104.01290/) (2021)
- [Towards Collaborative Question Answering: A Preliminary Study](https://mlsanity.com/paper/2201.09708/) (2022)
- [Better Query Graph Selection for Knowledge Base Question Answering](https://mlsanity.com/paper/2204.12662/) (2022)
- [A Continual Relation Extraction Approach for Knowledge Graph Completeness](https://mlsanity.com/paper/2404.17593/) (2024)
- [RFKG-CoT: Relation-Driven Adaptive Hop-count Selection and Few-Shot Path Guidance for Knowledge-Aware QA](https://mlsanity.com/paper/2512.15219/) (2025)

## Abstract (arXiv)

The SimpleQuestions dataset is one of the most commonly used benchmarks for studying single-relation factoid questions. In this paper, we present new evidence that this benchmark can be nearly solved by standard methods. First we show that ambiguity in the data bounds performance on this benchmark at 83.4%; there are often multiple answers that cannot be disambiguated from the linguistic signal alone. Second we introduce a baseline that sets a new state-of-the-art performance level at 78.1% accuracy, despite using standard methods. Finally, we report an empirical analysis showing that the upperbound is loose; roughly a third of the remaining errors are also not resolvable from the linguistic signal. Together, these results suggest that the SimpleQuestions dataset is nearly solved.