mlsanityData to 7 Oct 2026

On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models

arXiv 2406.16367 ↗First posted 24 Jun 2024cs.IRcited by 10

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval augmented generation (RAG) exhibits outstanding performance in promoting the knowledge capabilities of large language models (LLMs) with retrieved documents related to user queries. However, RAG only focuses on improving the response quality of LLMs via enhancing queries indiscriminately with retrieved information, paying little attention to what type of knowledge LLMs really need to answer original queries more accurately. In this paper, we suggest that long-tail knowledge is crucial for RAG as LLMs have already remembered common world knowledge during large-scale pre-training. Based on our observation, we propose a simple but effective long-tail knowledge detection method for LLMs. Specifically, the novel Generative Expected Calibration Error (GECE) metric is derived to measure the ``long-tailness'' of knowledge based on both statistics and semantics. Hence, we retrieve relevant documents and infuse them into the model for patching knowledge loopholes only when the input query relates to long-tail knowledge. Experiments show that, compared to existing RAG pipelines, our method achieves over 4x speedup in average inference time and consistent performance improvement in downstream tasks.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 31 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

21 of 31 references.

References · 21

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  8. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  9. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  10. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  11. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  12. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
9 more
  1. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  2. 2023Learning to Filter Context for Retrieval-Augmented Generationcited by 82
  3. 2023UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluationcited by 74
  4. 2023Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Studycited by 63
  5. 2021Towards Calibrated Model for Long-Tailed Visual Recognition from Prior Perspectivecited by 39
  6. 2023FEND: A Future Enhanced Distribution-Aware Contrastive Learning Framework for Long-tail Trajectory Predictioncited by 31
  7. 2022Balanced Product of Calibrated Experts for Long-Tailed Recognitioncited by 20
  8. 2021Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphscited by 9
  9. 2022GradTail: Learning Long-Tailed Data Using Gradient-based Sample Weightingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 3.

10 papers.

  1. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  2. 2024UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generationcited by 14
  3. 2024Parse Trees Guided LLM Prompt Compressioncited by 6
  4. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  5. 2025LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluationcited by 4
  6. 2024QCG-Rerank: Chunks Graph Rerank with Query Expansion in Retrieval-Augmented LLMs for Tourism Domaincited by 1
  7. 2025Efficiently Estimating Data Efficiency for Language Model Fine-tuningcited by 1
  8. 2026Latent Knowledge as a Predictor of Fact Acquisition in Fine-Tuned Large Language Modelscited by 0
  9. 2026Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognitioncited by 0
  10. 2026Domain-Adapted Retrieval for In-Context Annotation of Pedagogical Dialogue Actscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  2. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Survey
  4. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall
  5. 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective
  6. 2024KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models
  7. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  8. 2024Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks