mlsanityData to 7 Oct 2026

Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases

arXiv 2011.07743 ↗First posted 16 Nov 2020cs.CL cs.AI cs.LGcited by 169

Not on the map. Why?

Cited by 30 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing studies on question answering on knowledge bases (KBQA) mainly operate with the standard i.i.d assumption, i.e., training distribution over questions is the same as the test distribution. However, i.i.d may be neither reasonably achievable nor desirable on large-scale KBs because 1) true user distribution is hard to capture and 2) randomly sample training examples from the enormous space would be highly data-inefficient. Instead, we suggest that KBQA models should have three levels of built-in generalization: i.i.d, compositional, and zero-shot. To facilitate the development of KBQA models with stronger generalization, we construct and release a new large-scale, high-quality dataset with 64,331 questions, GrailQA, and provide evaluation settings for all three levels of generalization. In addition, we propose a novel BERT-based KBQA model. The combination of our dataset and model enables us to thoroughly examine and demonstrate, for the first time, the key role of pre-trained contextual embeddings like BERT in the generalization of KBQA.

Key facts. Cited by 169 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 20; 2023: 40; 2024: 35; 2025: 41; 2026: 21.

169 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  3. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  4. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  5. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  6. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  7. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  8. 2024KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graphcited by 90
  9. 2021RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answeringcited by 86
  10. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  11. 2022DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Basescited by 83
  12. 2021A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutionscited by 77
38 more
  1. 2024Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphscited by 77
  2. 2023Few-shot In-context Learning for Knowledge Base Question Answeringcited by 74
  3. 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agentscited by 72
  4. 2022Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answeringcited by 71
  5. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  6. 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generationcited by 67
  7. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 65
  8. 2025PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Pathscited by 60
  9. 2022TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Basescited by 59
  10. 2022UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Trainingcited by 57
  11. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  12. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  13. 2021Complex Knowledge Base Question Answering: A Surveycited by 52
  14. 2023FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answeringcited by 51
  15. 2022ArcaneQA: Dynamic Program Induction and Contextualized Encoding for Knowledge Base Question Answeringcited by 48
  16. 2023ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Modelscited by 44
  17. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  18. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environmentscited by 39
  19. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  20. 2022Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsingcited by 30
  21. 2023QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operationscited by 30
  22. 2024StructLM: Towards Building Generalist Models for Structured Knowledge Groundingcited by 29
  23. 2021Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalizationcited by 27
  24. 2024SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questionscited by 27
  25. 2024Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Modelscited by 25
  26. 2024Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Taskscited by 24
  27. 2022Knowledge Base Question Answering: A Semantic Parsing Perspectivecited by 22
  28. 2023FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answeringcited by 21
  29. 2023DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AIcited by 21
  30. 2023Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generationcited by 21
  31. 2023Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoningcited by 21
  32. 2024Atomic Fact Decomposition Helps Attributed Question Answeringcited by 21
  33. 2025KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Searchcited by 21
  34. 2023An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answeringcited by 19
  35. 2023Learning to Simulate Natural Language Feedback for Interactive Semantic Parsingcited by 19
  36. 2023DBLP-QuAD: A Question Answering Dataset over the DBLP Scholarly Knowledge Graphcited by 18
  37. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  38. 2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17

50 most cited of 169 All 169 (JSON)