mlsanityData to 14 Oct 2026

SymLoc: Symbolic Localization of Hallucination across HaluEval and TruthfulQA

arXiv 2511.14172 ↗First posted 18 Nov 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

LLMs still struggle with hallucination, especially when confronted with symbolic triggers like modifiers, negation, numbers, exceptions, and named entities. Yet, we lack a clear understanding of where these symbolic hallucinations originate, making it crucial to systematically handle such triggers and localize the emergence of hallucination inside the model. While prior work explored localization using statistical techniques like LSC and activation variance analysis, these methods treat all tokens equally and overlook the role symbolic linguistic knowledge plays in triggering hallucinations. So far, no approach has investigated how symbolic elements specifically drive hallucination failures across model layers, nor has symbolic linguistic knowledge been used as the foundation for a localization framework. We propose the first symbolic localization framework that leverages symbolic linguistic and semantic knowledge to meaningfully trace the development of hallucinations across all model layers. By focusing on how models process symbolic triggers, we analyze five models using HaluEval and TruthfulQA. Our symbolic knowledge approach reveals that attention variance for these linguistic elements explodes to critical instability in early layers (2-4), with negation triggering catastrophic variance levels, demonstrating that symbolic semantic processing breaks down from the very beginning. Through the lens of symbolic linguistic knowledge, despite larger model sizes, hallucination rates remain consistently high (78.3%-83.7% across Gemma variants), with steep attention drops for symbolic semantic triggers throughout deeper layers. Our findings demonstrate that hallucination is fundamentally a symbolic linguistic processing failure, not a general generation problem, revealing that symbolic semantic knowledge provides the key to understanding and localizing hallucination mechanisms in LLMs.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 18 Nov 2025.

Builds on

20 of 33 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  6. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  7. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  8. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  9. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  10. 2023Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sourcescited by 122
  11. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  12. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
8 more
  1. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformationscited by 20
  2. 2024Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretabilitycited by 13
  3. 2023Learning to Paraphrase Sentences to Different Complexity Levelscited by 11
  4. 2024WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensionscited by 10
  5. 2024A Domain-Agnostic Neurosymbolic Approach for Big Social Data Analysis: Evaluating Mental Health Sentiment on Social Media during COVID-19cited by 5
  6. 2024Layer Importance and Hallucination Analysis in Large Language Models via Enhanced Activation Variance-Sparsitycited by 5
  7. 2025Investigating Symbolic Triggers of Hallucination in Gemma Models Across HaluEval and TruthfulQAcited by 3
  8. 2025Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Trainingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Preventioncited by 0