mlsanityData to 7 Oct 2026

Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation

arXiv 2411.07773 ↗First posted 12 Nov 2024cs.CL cs.AI cs.LGcited by 6

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Recent work suggests that large language models enhanced with retrieval-augmented generation are easily influenced by the order, in which the retrieved documents are presented to the model when solving tasks such as question answering (QA). However, there is no method to date that exploits this phenomenon to improve generation. We fill this gap. In this study, we show that the pointwise mutual information between a context and a question is an effective gauge for language model performance. Importantly, this gauge does not depend on knowing the answer to the question a priori. Through experiments on two question-answering datasets and a variety of large language models, we find evidence for an empirical correlation between answer accuracy and pointwise mutual information. Additionally, we propose two methods that use the pointwise mutual information between a document and a question as a gauge for selecting and constructing prompts that lead to better performance, whose effectiveness we demonstrate through experimentation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 26 of its 34 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

26 of 34 references.

References · 26

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
14 more
  1. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  3. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  4. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  5. 2019ELI5: Long Form Question Answeringcited by 567
  6. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  7. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  8. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  9. 2020How Context Affects Language Models' Factual Predictionscited by 181
  10. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  11. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  12. 2021One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrievalcited by 46
  13. 2013Context-Specific Independence in Bayesian Networkscited by 43
  14. 2024Crafting Knowledge: Exploring the Creative Mechanisms of Chat-Based Search Enginescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottleneckscited by 4
  2. 2026Memory Retrieval for Changing Preferencescited by 3
  3. 2026Revisiting RAG Retrievers: An Information Theoretic Benchmarkcited by 2
  4. 2025Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaborationcited by 1
  5. 2025Directed Information γ-covering: An Information-Theoretic Framework for Context Engineeringcited by 1
  6. 2025Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
  2. 2024Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis
  3. 2025Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
  4. 2025LLM-Specific Utility for Retrieval-Augmented Generation
  5. 2025Retrieval Augmented Question Answering: When Should LLMs Admit Ignorance?
  6. 2023Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs
  7. 2026Predicting the Benefit of Retrieval Augmentation in Open-Domain Question Answering
  8. 2023Retrieval-Generation Synergy Augmented Large Language Models