mlsanityData to 7 Oct 2026

MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations

arXiv 2506.01367 ↗First posted 2 Jun 2025cs.CL stat.MLcited by 0

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Large Language Models (LLMs) are increasingly integrated into agentic AI systems, yet their propensity to generate hallucinations remains a critical safety concern. Detecting these factual errors at test-time, particularly without ground-truth labels, is essential for building trustworthy autonomous agents. We propose MMD-Flagger, an hallucination detection method that utilizes Maximum Mean Discrepancy (MMD) and monitors the stability of LLM outputs across varying decoding temperatures. Our method tracks the MMD trajectory between a LLM's response at a certain decoding configuration and a set of stochastic samples, identifying hallucinations based on the trajectory's characteristic shape. We evaluate MMDFlagger on multi-lingual claim verification benchmarks (MUCH) using modern LLMs like Llama-3 families and Gemma-3.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 59 of its 84 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

59 of 84 references.

References · 59

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2024Qwen2 Technical Reportcited by 2,076
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  11. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  12. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
38 more
  1. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  2. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  3. 2018Hierarchical Neural Story Generationcited by 1,182
  4. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  5. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  6. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  7. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  8. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 360
  9. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  10. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  11. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  12. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 210
  13. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  14. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  15. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  16. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
  17. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  18. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  19. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  20. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  21. 2020If beam search is the answer, what was the question?cited by 114
  22. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 112
  23. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  24. 2016Generative Models and Model Criticism via Optimized Maximum Mean Discrepancycited by 99
  25. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  26. 2024Is Temperature the Creativity Parameter of Large Language Models?cited by 92
  27. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 84
  28. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  29. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  30. 2024Calibrating Language Models with Adaptive Temperature Scalingcited by 57
  31. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 50
  32. 2025Steer LLM Latents for Hallucination Detectioncited by 49
  33. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  34. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  35. 2025Hallucination Detection in LLMs Using Spectral Features of Attention Mapscited by 42
  36. 2020Multi-Loss Weighting with Coefficient of Variationscited by 36
  37. 2024InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answerscited by 26
  38. 2024FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMscited by 24

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique
  2. 2025Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor
  3. 2025HalluField: Detecting LLM Hallucinations via Field-Theoretic Modeling
  4. 2026Zero-source LLM Hallucination Detection with Human-like Criteria Probing
  5. 2024Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
  6. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
  7. 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
  8. 2026Scalable Token-Level Hallucination Detection in Large Language Models