# HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs

arXiv: https://arxiv.org/abs/2506.00088 · HTML: https://mlsanity.com/paper/2506.00088/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 28 of 28)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Long-form factuality in large language models](https://mlsanity.com/paper/2403.18802/) (2024)
- [Graph Neural Controlled Differential Equations for Traffic Forecasting](https://mlsanity.com/paper/2112.03558/) (2021)
- [Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View](https://mlsanity.com/paper/1906.02762/) (2019)
- [Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus](https://mlsanity.com/paper/2311.13230/) (2023)
- [Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks](https://mlsanity.com/paper/2307.00175/) (2023)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [Multi-level Residual Networks from Dynamical Systems View](https://mlsanity.com/paper/1710.10348/) (2017)
- [Neural Controlled Differential Equations for Online Prediction Tasks](https://mlsanity.com/paper/2106.11028/) (2021)
- [Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems](https://mlsanity.com/paper/2109.15142/) (2021)
- [Attentive Neural Controlled Differential Equations for Time-series Classification and Forecasting](https://mlsanity.com/paper/2109.01876/) (2021)
- [ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation](https://mlsanity.com/paper/2203.09176/) (2022)
- [PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics](https://mlsanity.com/paper/2404.04722/) (2024)

## Built on by (most cited first, 6 of 6)

- [AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents](https://mlsanity.com/paper/2601.06818/) (2026)
- [Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs](https://mlsanity.com/paper/2601.11061/) (2026)
- [MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing](https://mlsanity.com/paper/2605.24919/) (2026)
- [When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs](https://mlsanity.com/paper/2511.07318/) (2025)
- [Decoding Memories: An Efficient Pipeline for Self-Consistency Hallucination Detection](https://mlsanity.com/paper/2508.21228/) (2025)
- [CausalGaze: Unveiling Hallucinations via Counterfactual Graph Intervention in Large Language Models](https://mlsanity.com/paper/2604.11087/) (2026)

## Abstract (arXiv)

In recent years, large language models (LLMs) have made remarkable advancements, yet hallucination, where models produce inaccurate or non-factual statements, remains a significant challenge for real-world deployment. Although current classification-based methods, such as SAPLMA, are highly efficient in mitigating hallucinations, they struggle when non-factual information arises in the early or mid-sequence of outputs, reducing their reliability. To address these issues, we propose Hallucination Detection-Neural Differential Equations (HD-NDEs), a novel method that systematically assesses the truthfulness of statements by capturing the full dynamics of LLMs within their latent space. Our approaches apply neural differential equations (Neural DEs) to model the dynamic system in the latent space of LLMs. Then, the sequence in the latent space is mapped to the classification space for truth assessment. The extensive experiments across five datasets and six widely used LLMs demonstrate the effectiveness of HD-NDEs, especially, achieving over 14% improvement in AUC-ROC on the True-False dataset compared to state-of-the-art techniques.