mlsanityData to 7 Oct 2026

KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base

arXiv 2007.03875 ↗First posted 8 Jul 2020cs.CLcited by 76

Not on the map. Why?

Abstract (arXiv)

Complex question answering over knowledge base (Complex KBQA) is challenging because it requires various compositional reasoning capabilities, such as multi-hop inference, attribute comparison, set operation. Existing benchmarks have some shortcomings that limit the development of Complex KBQA: 1) they only provide QA pairs without explicit reasoning processes; 2) questions are poor in diversity or scale. To this end, we introduce KQA Pro, a dataset for Complex KBQA including ~120K diverse natural language questions. We introduce a compositional and interpretable programming language KoPL to represent the reasoning process of complex questions. For each question, we provide the corresponding KoPL program and SPARQL query, so that KQA Pro serves for both KBQA and semantic parsing tasks. Experimental results show that SOTA KBQA methods cannot achieve promising results on KQA Pro as on current datasets, which suggests that KQA Pro is challenging and Complex KBQA requires further research efforts. We also treat KQA Pro as a diagnostic dataset for testing multiple reasoning skills, conduct a thorough evaluation of existing models and discuss further directions for Complex KBQA. Our codes and datasets can be obtained from https://github.com/shijx12/KQAPro_Baselines.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 22; 2024: 30; 2025: 5; 2026: 10.

76 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  3. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 121
  4. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  5. 2024KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graphcited by 90
  6. 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
  7. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  8. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  9. 2022TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Basescited by 59
  10. 2024STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Basescited by 53
  11. 2023FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answeringcited by 51
  12. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environmentscited by 39
38 more
  1. 2024SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questionscited by 27
  2. 2024Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Modelscited by 25
  3. 2024MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistantscited by 24
  4. 2024Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrievalcited by 24
  5. 2023ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graphcited by 23
  6. 2022Knowledge Base Question Answering: A Semantic Parsing Perspectivecited by 22
  7. 2023FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answeringcited by 21
  8. 2024SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graphcited by 19
  9. 2023DBLP-QuAD: A Question Answering Dataset over the DBLP Scholarly Knowledge Graphcited by 18
  10. 2024Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Artcited by 17
  11. 2024SportQA: A Benchmark for Sports Understanding in Large Language Modelscited by 15
  12. 2024A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practicescited by 15
  13. 2024Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Factscited by 14
  14. 2022GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representationcited by 13
  15. 2023Semantic Decomposition of Question and SQL for Text-to-SQL Parsingcited by 13
  16. 2024SECURE: Benchmarking Large Language Models for Cybersecuritycited by 13
  17. 2024CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Eracited by 13
  18. 2023Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answeringcited by 11
  19. 2023Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Modelscited by 11
  20. 2025Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarkingcited by 11
  21. 2024Large Language Models Can Better Understand Knowledge Graphs Than We Thoughtcited by 10
  22. 2021Compositional Generalization in Multilingual Semantic Parsing over Wikidatacited by 9
  23. 2024Temporal Knowledge Graph Question Answering: A Surveycited by 9
  24. 2024KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questionscited by 9
  25. 2022Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questionscited by 7
  26. 2023MarkQA: A large scale KBQA dataset with numerical reasoningcited by 6
  27. 2024Multi-hop Question Answering over Knowledge Graphs using Large Language Modelscited by 6
  28. 2024Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Videocited by 6
  29. 2025DEEPAMBIGQA: Ambiguous Multi-hop Questions for Benchmarking LLM Answer Completenesscited by 6
  30. 2022Financial data analysis application via multi-strategy text processingcited by 5
  31. 2022Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsingcited by 5
  32. 2024How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answeringcited by 5
  33. 2024Self-Improvement Programming for Temporal Knowledge Graph Question Answeringcited by 5
  34. 2024TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Datacited by 5
  35. 2022ReasonChainQA: Text-based Complex Question Answering with Explainable Evidence Chainscited by 4
  36. 2023Exploring & Exploiting High-Order Graph Structure for Sparse Knowledge Graph Completioncited by 4
  37. 2023Knowledge Base Question Answering for Space Debris Queriescited by 3
  38. 2023Data Distribution Bottlenecks in Grounding Language Models to Knowledge Basescited by 3

50 most cited of 76 All 76 (JSON)