mlsanityData to 14 Oct 2026

Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92%

arXiv 2512.03107 ↗First posted 2 Dec 2025cs.LG cs.CL q-fin.CPcited by 2

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) produce fluent but unsupported answers - hallucinations - limiting safe deployment in high-stakes domains. We propose ECLIPSE, a framework that treats hallucination as a mismatch between a model's semantic entropy and the capacity of available evidence. We combine entropy estimation via multi-sample clustering with a novel perplexity decomposition that measures how models use retrieved evidence. We prove that under mild conditions, the resulting entropy-capacity objective is strictly convex with a unique stable optimum. We evaluate on a controlled financial question answering dataset with GPT-3.5-turbo (n=200 balanced samples with synthetic hallucinations), where ECLIPSE achieves ROC AUC of 0.89 and average precision of 0.90, substantially outperforming a semantic entropy-only baseline (AUC 0.50). A controlled ablation with Claude-3-Haiku, which lacks token-level log probabilities, shows AUC dropping to 0.59 with coefficient magnitudes decreasing by 95% - demonstrating that ECLIPSE is a logprob-native mechanism whose effectiveness depends on calibrated token-level uncertainties. The perplexity decomposition features exhibit the largest learned coefficients, confirming that evidence utilization is central to hallucination detection. We position this work as a controlled mechanism study; broader validation across domains and naturally occurring hallucinations remains future work.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 31 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

28 of 31 references.

References · 28

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  10. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  11. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  12. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
16 more
  1. 2023BloombergGPT: A Large Language Model for Financecited by 939
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  3. 2017Selective Classification for Deep Neural Networkscited by 852
  4. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  6. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  7. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  9. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  10. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  11. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  12. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  13. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  14. 2020A Theory of Usable Information Under Computational Constraintscited by 162
  15. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 138
  16. 2022Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settingscited by 66

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.