mlsanityData to 13 Oct 2026

Estimating the Hallucination Rate of Generative AI

arXiv 2406.07457 ↗First posted 11 Jun 2024cs.LG stat.MLcited by 20

Not on the map. Why?

Abstract (arXiv)

This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 69 of its 101 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

69 of 101 references.

References · 69

  1. 2017Attention Is All You Needcited by 60,106
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2023GPT-4 Technical Reportcited by 18,278
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,480
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,197
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,319
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,823
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,376
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,280
  12. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
38 more
  1. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,126
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,042
  3. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 925
  4. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 898
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 810
  6. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 798
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 750
  8. 2022Teaching Models to Express Their Uncertainty in Wordscited by 740
  9. 2024Chronos: Learning the Language of Time Seriescited by 737
  10. 2022Transformers learn in-context by gradient descentcited by 720
  11. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 693
  12. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 676
  13. 2023The Internal State of an LLM Knows When It's Lyingcited by 652
  14. 2018Conditional Neural Processescited by 423
  15. 2023Larger language models do in-context learning differentlycited by 387
  16. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 374
  17. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  18. 2021Transformers Can Do Bayesian Inferencecited by 346
  19. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 324
  20. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 320
  21. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  22. 2018Neural Processescited by 309
  23. 2023In-Context Learning Creates Task Vectorscited by 305
  24. 2023Function Vectors in Large Language Modelscited by 299
  25. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 289
  26. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  27. 2019Attentive Neural Processescited by 281
  28. 2023Fine-tuning Language Models for Factualitycited by 219
  29. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  30. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 205
  31. 2024Many-Shot In-Context Learningcited by 205
  32. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  33. 2018Hate Speech Dataset from a White Supremacy Forumcited by 198
  34. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 151
  35. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  36. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 126
  37. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 124
  38. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 122

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 6.

20 papers.

Sort:

By area:

  1. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 40
  2. 2025RAG-KG-IL: A Multi-Agent Hybrid Framework for Reducing Hallucinations and Enhancing LLM Reasoning through RAG and Incremental Knowledge Graph Learning Integrationcited by 9
  3. 2025Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based Systemscited by 8
  4. 2025Do Large Language Models (Really) Need Statistical Foundations?cited by 5
  5. 2025Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generationcited by 4
  6. 2025LLM-to-Phy3D: Physically Conform Online 3D Object Generation with LLMscited by 4
  7. 2025Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddingscited by 3
  8. 2024Can Generative AI Solve Your In-Context Learning Problem? A Martingale Perspectivecited by 2
  9. 2025Balancing Content Size in RAG-Text2SQL Systemcited by 2
  10. 2025Measuring the Impact of Lexical Training Data Coverage on Hallucination Detection in Large Language Modelscited by 2
  11. 2025Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layerscited by 2
  12. 2026Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Modelscited by 2
8 more
  1. 2024Rejecting Hallucinated State Targets during Planningcited by 1
  2. 2025CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processingcited by 1
  3. 2025Machine Mirages: Defining the Undefinedcited by 0
  4. 2026In-Context Function Learning in Large Language Modelscited by 0
  5. 2026Referring Multiple Regions with Large Multimodal Models via Contextual Latent Steeringcited by 0
  6. 2026TIGER: Text-Informed Generalized Enzyme-Reaction Retrievalcited by 0
  7. 2026What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustnesscited by 0
  8. 2026The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstainingcited by 0