Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Models
On the map: LLM reasoning → Hallucination detection and mitigation
Abstract (arXiv)
Hallucination detection in large language models (LLMs) is deployment-critical, and recent work shows that the spectrum of attention-derived graph Laplacians carries strong signal about reasoning quality. Prior spectral diagnostics, however, summarize the Laplacian spectrum by a handful of eigenvalues or hand-picked scalars, leaving most of its structure unused. We propose Free-Energy Signatures (Fes), a spectral descriptor that treats each layer's attention Laplacian as a Hamiltonian and extracts its thermodynamic potentials partition function, free energy, spectral entropy, heat capacity together with the random-matrix-theory (RMT) spectral form factor. We prove three results: (i)~Lipschitz stability of Fes under attention perturbation; (ii)~an expressiveness result showing that Fes enriches finite spectral summaries and approximates moment-derived spectral functionals under explicit regularity and grid-resolution assumptions; and (iii)~a finite-sample PAC bound on the AUROC of a training-free detector built from Fes. Empirically, across six open-weight LLMs and six benchmarks, a lightweight probe on Fes descriptors achieves the strongest aggregate AUROC among attention-spectral baselines, improving over LapEig by +6.5 AUROC points and over GoR-4 by +2.4 points on average, while requiring no update to the underlying LLM. In the fully unsupervised setting, an RMT-deviation score achieves mean AUROC 0.71, providing a label-free but weaker detector. A complementary RMT analysis shows that correct generations exhibit more Wigner-Dyson like spectral statistics, whereas hallucinations exhibit more Poisson-like statistics. The anonymized code and config are provided in the supplementary material.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 12 of its 27 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
12 of 27 references.
References · 12
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2023Let's Verify Step by Stepcited by 3,789
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
- 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 290
- 2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 215
- 2025Hallucination Detection in LLMs Using Spectral Features of Attention Mapscited by 43
- 2025ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMscited by 32
- 2018The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Sizecited by 13
- 2026Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoningcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
- 2025HalluField: Detecting LLM Hallucinations via Field-Theoretic Modeling
- 2025Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models
- 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
- 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphs
- 2026The Energy of Falsehood: Detecting Hallucinations via Diffusion Model Likelihoods
- 2026TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs
- 2025Hallucination Detection via Internal States and Structured Reasoning Consistency in Large Language Models