mlsanityData to 7 Oct 2026

FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation

arXiv 2209.14290 ↗First posted 28 Sep 2022cs.CL cs.IRcited by 63

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-augmented generation models offer many benefits over standalone language models: besides a textual answer to a given query they provide provenance items retrieved from an updateable knowledge base. However, they are also more complex systems and need to handle long inputs. In this work, we introduce FiD-Light to strongly increase the efficiency of the state-of-the-art retrieval-augmented FiD model, while maintaining the same level of effectiveness. Our FiD-Light model constrains the information flow from the encoder (which encodes passages separately) to the decoder (using concatenated encoded representations). Furthermore, we adapt FiD-Light with re-ranking capabilities through textual source pointers, to improve the top-ranked provenance precision. Our experiments on a diverse set of seven knowledge intensive tasks (KILT) show FiD-Light consistently improves the Pareto frontier between query latency and effectiveness. FiD-Light with source pointing sets substantial new state-of-the-art results on six KILT tasks for combined text generation and provenance retrieval evaluation, while maintaining reasonable efficiency.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 48 of its 57 references that are in the map. First posted on arXiv 28 Sep 2022.

Builds on

48 of 57 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
36 more
  1. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  2. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  3. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  5. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  6. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  7. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  8. 2022Confident Adaptive Language Modelingcited by 297
  9. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  10. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  11. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  12. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  13. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  14. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  15. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  16. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  17. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
  18. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  19. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  20. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  21. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  22. 2019CEDR: Contextualized Embeddings for Document Rankingcited by 77
  23. 2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 66
  24. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  25. 2020QED: A Framework and Dataset for Explanations in Question Answeringcited by 55
  26. 2022Retrieval-Enhanced Machine Learningcited by 46
  27. 2021Multi-task Retrieval for Knowledge-Intensive Taskscited by 44
  28. 2021Evidentiality-guided Generation for Knowledge-Intensive NLP Taskscited by 40
  29. 2021Hindsight: Posterior-guided training of retrievers for improved open-ended generationcited by 38
  30. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  31. 2021Robust Retrieval Augmented Generation for Zero-shot Slot Fillingcited by 24
  32. 2019What Question Answering can Learn from Trivia Nerdscited by 21
  33. 2020FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generationcited by 21
  34. 2020DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modelingcited by 14
  35. 2022Multi-Task Retrieval-Augmented Text Generation with Relevance Samplingcited by 10
  36. 2022Are We There Yet? A Decision Framework for Replacing Term Based Retrieval with Dense Retrieval Systemscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 14; 2024: 28; 2025: 15; 2026: 3.

63 papers.

Sort:

By area:

  1. 2023A Comprehensive Overview of Large Language Modelscited by 639
  2. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  3. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  4. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  5. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  6. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation Systemcited by 138
  7. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  8. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  9. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  10. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  11. 2025PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Pathscited by 60
  12. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
38 more
  1. 2022FiDO: Fusion-in-Decoder optimized for stronger performance and faster inferencecited by 31
  2. 2024Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximizationcited by 29
  3. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  4. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 26
  5. 2023Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesiscited by 25
  6. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  7. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
  8. 2023Unified Language Representation for Question Answering over Text, Tables, and Imagescited by 22
  9. 2022QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillationcited by 21
  10. 2024Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decodingcited by 19
  11. 2023Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Modelscited by 18
  12. 2023Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your computecited by 17
  13. 2024Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generationcited by 15
  14. 2023A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answeringcited by 14
  15. 2024Accelerating Retrieval-Augmented Generationcited by 14
  16. 2023GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Rankingcited by 11
  17. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  18. 2025MIRAGE: A Metric-Intensive Benchmark for Retrieval-Augmented Generation Evaluationcited by 11
  19. 2023Optimizing Retrieval-augmented Reader Models via Token Eliminationcited by 10
  20. 2024FT2Ra: A Fine-Tuning-Inspired Approach to Retrieval-Augmented Code Completioncited by 9
  21. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
  22. 2024PDF-MVQA: A Dataset for Multimodal Information Retrieval in PDF-based Visual Question Answeringcited by 7
  23. 2025Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generationcited by 7
  24. 2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
  25. 2026Query-Aware Flow Diffusion for Graph-Based RAG with Retrieval Guaranteescited by 7
  26. 2024Multimodal Reranking for Knowledge-Intensive Visual Question Answeringcited by 6
  27. 2023BTR: Binary Token Representations for Efficient Retrieval Augmented Language Modelscited by 5
  28. 2024Crafting Knowledge: Exploring the Creative Mechanisms of Chat-Based Search Enginescited by 5
  29. 2024Dynamic Demonstration Retrieval and Cognitive Understanding for Emotional Support Conversationcited by 5
  30. 2025LLM-based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verificationcited by 5
  31. 2025OSCAR: Online Soft Compression And Rerankingcited by 4
  32. 2025Leveraging LLM-Assisted Query Understanding for Live Retrieval-Augmented Generationcited by 4
  33. 2024Generalist Multimodal AI: A Review of Architectures, Challenges and Opportunitiescited by 3
  34. 2023Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selectioncited by 2
  35. 2024Exploring ChatGPT for Next-generation Information Retrieval: Opportunities and Challengescited by 2
  36. 2024Multi-Granularity Guided Fusion-in-Decodercited by 2
  37. 2024MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sourcescited by 2
  38. 2024FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selectioncited by 2

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Retrieval is Accurate Generation
  2. 2023Active Retrieval Augmented Generation
  3. 2024LightRAG: Simple and Fast Retrieval-Augmented Generation
  4. 2026Latent Abstraction for Retrieval-Augmented Generation
  5. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  6. 2026M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation
  7. 2025PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation
  8. 2025Beyond Perplexity: Let the Reader Select Retrieval Summaries via Spectrum Projection Score