# SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMs

arXiv: https://arxiv.org/abs/2503.03032 · HTML: https://mlsanity.com/paper/2503.03032/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination detection from model internals, one of 13 directions in Mechanistic interpretability.
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Placed in

- Mechanistic interpretability / [Hallucination detection from model internals](https://mlsanity.com/mechinterp/hallucination-detection-from-model-internals/)

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [Alleviating Hallucinations of Large Language Models through Induced Hallucinations](https://mlsanity.com/paper/2312.15710/) (2023)
- [RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations](https://mlsanity.com/paper/2402.17700/) (2024)
- [LLMs Will Always Hallucinate, and We Need to Live With This](https://mlsanity.com/paper/2409.05746/) (2024)
- [UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation](https://mlsanity.com/paper/2303.08518/) (2023)
- [Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning](https://mlsanity.com/paper/2312.17484/) (2023)
- [Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs](https://mlsanity.com/paper/2410.11179/) (2024)

## Built on by (most cited first, 8 of 8)

- [Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs](https://mlsanity.com/paper/2505.20254/) (2025)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders](https://mlsanity.com/paper/2512.08892/) (2025)
- [HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoders](https://mlsanity.com/paper/2604.16430/) (2026)
- [NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels](https://mlsanity.com/paper/2603.02219/) (2026)
- [Interpretability Transfer from Language to Vision via Sparse Autoencoders](https://mlsanity.com/paper/2605.24946/) (2026)
- [Is Monitoring Enough? Strategic Agent Selection For Stealthy Attack in Multi-Agent Discussions](https://mlsanity.com/paper/2603.21194/) (2026)
- [Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility](https://mlsanity.com/paper/2610.20139/) (2026)

## Abstract (arXiv)

Despite the state-of-the-art performance of Large Language Models (LLMs), these models often suffer from hallucinations, which can undermine their performance in critical applications. In this work, we propose SAFE, a novel method for detecting and mitigating hallucinations by leveraging Sparse Autoencoders (SAEs). While hallucination detection techniques and SAEs have been explored independently, their synergistic application in a comprehensive system, particularly for hallucination-aware query enrichment, has not been fully investigated. To validate the effectiveness of SAFE, we evaluate it on two models with available SAEs across three diverse cross-domain datasets designed to assess hallucination problems. Empirical results demonstrate that SAFE consistently improves query generation accuracy and mitigates hallucinations across all datasets, achieving accuracy improvements of up to 29.45%.