mlsanityData to 7 Oct 2026

SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach

arXiv 1804.08798 ↗First posted 24 Apr 2018cs.CL cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

The SimpleQuestions dataset is one of the most commonly used benchmarks for studying single-relation factoid questions. In this paper, we present new evidence that this benchmark can be nearly solved by standard methods. First we show that ambiguity in the data bounds performance on this benchmark at 83.4%; there are often multiple answers that cannot be disambiguated from the linguistic signal alone. Second we introduce a baseline that sets a new state-of-the-art performance level at 78.1% accuracy, despite using standard methods. Finally, we report an empirical analysis showing that the upperbound is loose; roughly a third of the remaining errors are also not resolvable from the linguistic signal. Together, these results suggest that the SimpleQuestions dataset is nearly solved.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 2; 2023: 3; 2024: 3; 2025: 2.

18 papers.

Sort:

By area:

  1. 2021TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graphcited by 60
  2. 2021Complex Knowledge Base Question Answering: A Surveycited by 52
  3. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  4. 2023An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answeringcited by 19
  5. 2021UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Textcited by 17
  6. 2021End-to-End Entity Resolution and Question Answering Using Differentiable Knowledge Graphscited by 11
  7. 2021Grounding Dialogue Systems via Knowledge Graph Aware Decoding with Pre-trained Transformerscited by 8
  8. 2021Improving Embedded Knowledge Graph Multi-hop Question Answering by introducing Relational Chain Reasoningcited by 7
  9. 2023Data Distribution Bottlenecks in Grounding Language Models to Knowledge Basescited by 3
  10. 2024Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answeringcited by 3
  11. 2023Two is Better Than One: Answering Complex Questions by Multiple Knowledge Sources with Generalized Linkscited by 2
  12. 2021Knowledge Graph Question Answering via SPARQL Silhouette Generationcited by 1
6 more
  1. 2025Plan Then Retrieve: Reinforcement Learning-Guided Complex Reasoning over Knowledge Graphscited by 1
  2. 2021Measuring Linguistic Diversity During COVID-19cited by 0
  3. 2022Towards Collaborative Question Answering: A Preliminary Studycited by 0
  4. 2022Better Query Graph Selection for Knowledge Base Question Answeringcited by 0
  5. 2024A Continual Relation Extraction Approach for Knowledge Graph Completenesscited by 0
  6. 2025RFKG-CoT: Relation-Driven Adaptive Hop-count Selection and Few-Shot Path Guidance for Knowledge-Aware QAcited by 0