# AFTER: Mitigating the Object Hallucination of LVLM via Adaptive Factual-Guided Activation Editing

arXiv: https://arxiv.org/abs/2601.01957 · HTML: https://mlsanity.com/paper/2601.01957/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 5 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)
- Mechanistic interpretability / [Hallucination detection from model internals](https://mlsanity.com/mechinterp/hallucination-detection-from-model-internals/)

## Built on by (most cited first, 4 of 4)

- [Test-Time Hallucination Control in Large Vision-Language Models](https://mlsanity.com/paper/2608.11474/) (2026)
- [Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods](https://mlsanity.com/paper/2609.01888/) (2026)
- [From Token Correction to Claim Admission: Training-Free Hallucination Mitigation for Large Vision-Language Models](https://mlsanity.com/paper/2610.17706/) (2026)
- [Mitigating Video Hallucinations via Attention-Guided Representation Perturbation](https://mlsanity.com/paper/2610.19728/) (2026)

## Abstract (arXiv)

Large Vision-Language Models (LVLMs) have achieved substantial progress in cross-modal tasks. However, due to language bias, LVLMs are susceptible to object hallucination, which can be primarily divided into category, attribute, and relation hallucination, significantly impeding the trustworthy AI applications. Editing the internal activations of LVLMs has shown promising effectiveness in mitigating hallucinations with minimal cost. However, previous editing approaches neglect the effective guidance offered by factual textual semantics, thereby struggling to explicitly mitigate language bias. To address these issues, we propose Adaptive Factual-guided Visual-Textual Editing for hallucination mitigation (AFTER), which comprises Factual-Augmented Activation Steering (FAS) and Query-Adaptive Offset Optimization (QAO), to adaptively guides the original biased activations towards factual semantics. Specifically, FAS is proposed to provide factual and general guidance for activation editing, thereby explicitly modeling the precise visual-textual associations. Subsequently, QAO introduces a query-aware offset estimator to establish query-specific editing from the general steering vector, enhancing the diversity and granularity of editing. Extensive experiments on standard hallucination benchmarks across three widely adopted LVLMs validate the efficacy of the proposed AFTER, notably achieving up to a 16.3% reduction of hallucination over baseline on the AMBER benchmark. Our code and data will be released for reproducibility.