mlsanityData to 7 Oct 2026

KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering

arXiv 2110.04330 ↗First posted 8 Oct 2021cs.CL cs.LGcited by 66

Not on the map. Why?

Abstract (arXiv)

Current Open-Domain Question Answering (ODQA) model paradigm often contains a retrieving module and a reading module. Given an input question, the reading module predicts the answer from the relevant passages which are retrieved by the retriever. The recent proposed Fusion-in-Decoder (FiD), which is built on top of the pretrained generative model T5, achieves the state-of-the-art performance in the reading module. Although being effective, it remains constrained by inefficient attention on all retrieved passages which contain a lot of noise. In this work, we propose a novel method KG-FiD, which filters noisy passages by leveraging the structural relationship among the retrieved passages with a knowledge graph. We initiate the passage node embedding from the FiD encoder and then use graph neural network (GNN) to update the representation for reranking. To improve the efficiency, we build the GNN on top of the intermediate layer output of the FiD encoder and only pass a few top reranked passages into the higher layers of encoder and decoder for answer generation. We also apply the proposed GNN based reranking method to enhance the passage retrieval results in the retrieving module. Extensive experiments on common ODQA benchmark datasets (Natural Question and TriviaQA) demonstrate that KG-FiD can improve vanilla FiD by up to 1.5% on answer exact match score and achieve comparable performance with FiD with only 40% of computation cost.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

35 of 53 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2017Graph Attention Networkscited by 8,854
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
23 more
  1. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  3. 2018Deep Graph Infomaxcited by 1,101
  4. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  5. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  6. 2019Passage Re-ranking with BERTcited by 629
  7. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  8. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  9. 2019PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Textcited by 182
  10. 2018Open Domain Question Answering Using Early Fusion of Knowledge Bases and Textcited by 167
  11. 2019End-to-End Open-Domain Question Answering with BERTserinicited by 152
  12. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  13. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  14. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  15. 2020JAKET: Joint Pre-training of Knowledge Graph and Language Understandingcited by 73
  16. 2019Knowledge Guided Text Retrieval and Reading for Open Domain Question Answeringcited by 59
  17. 2021UnitedQA: A Hybrid Approach for Open Domain Question Answeringcited by 42
  18. 2019Improving Question Answering over Incomplete KBs with Knowledge-Aware Readercited by 37
  19. 2020Fusing Context Into Knowledge Graph for Commonsense Question Answeringcited by 33
  20. 2021Rider: Reader-Guided Passage Reranking for Open-Domain Question Answeringcited by 28
  21. 2021Does Knowledge Help General NLU? An Empirical Studycited by 7
  22. 2021Pruning the Index Contents for Memory Efficient Open-Domain QAcited by 6
  23. 2020Knowledge-Aided Open-Domain Question Answeringcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 19; 2023: 27; 2024: 12; 2025: 4; 2026: 1.

66 papers.

Sort:

By area:

  1. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  3. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  4. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  5. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  6. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  7. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  8. 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answeringcited by 97
  9. 2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
  10. 2023CoLT5: Faster Long-Range Transformers with Conditional Computationcited by 84
  11. 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
  12. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
38 more
  1. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  2. 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 65
  3. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generationcited by 63
  4. 2022Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Datacited by 61
  5. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  6. 2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
  7. 2023Scientific Fact-Checking: A Survey of Resources and Approachescited by 53
  8. 2024Don't Forget to Connect! Improving RAG with Graph-based Rerankingcited by 49
  9. 2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
  10. 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
  11. 2021Dict-BERT: Enhancing Language Model Pre-training with Dictionarycited by 38
  12. 2023Evaluating Open-QA Evaluationcited by 35
  13. 2022FiDO: Fusion-in-Decoder optimized for stronger performance and faster inferencecited by 31
  14. 2022Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generationcited by 28
  15. 2022SKILL: Structured Knowledge Infusion for Large Language Modelscited by 27
  16. 2023Large Language Models are Built-in Autoregressive Search Enginescited by 26
  17. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25
  18. 2022Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answeringcited by 22
  19. 2023Fairness-Aware Graph Neural Networks: A Surveycited by 22
  20. 2022Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Expertscited by 19
  21. 2023Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your computecited by 17
  22. 2023Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQAcited by 17
  23. 2023ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Sciencecited by 15
  24. 2022Retrieval Augmentation for Commonsense Reasoning: A Unified Approachcited by 14
  25. 2022Empowering Language Models with Knowledge Graph Reasoning for Question Answeringcited by 14
  26. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 14
  27. 2023Large Knowledge Model: Perspectives and Challengescited by 12
  28. 2022KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understandingcited by 9
  29. 2023AVIS: Autonomous Visual Information Seeking with Large Language Model Agentcited by 8
  30. 2022A Unified Encoder-Decoder Framework with Entity Memorycited by 7
  31. 2023GLIMMER: generalized late-interaction memory rerankercited by 7
  32. 2024KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregationcited by 7
  33. 2022Question-Answer Sentence Graph for Joint Modeling Answer Selectioncited by 6
  34. 2022FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answeringcited by 5
  35. 2023Single Sequence Prediction over Reasoning Graphs for Multi-hop QAcited by 5
  36. 2023Construction contract risk identification based on knowledge-augmented language modelcited by 5
  37. 2025YABLoCo: Yet Another Benchmark for Long Context Code Generationcited by 5
  38. 2023Detecting Spoilers in Movie Reviews with External Movie Knowledge and User Networkscited by 4

50 most cited of 66 All 66 (JSON)