mlsanityData to 7 Oct 2026

Faithful Activation Verbalization: Reducing Hallucinations in LLM Representation Interpretation

arXiv 2609.34033 ↗First posted 27 Sep 2026cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG also fits: LLM preference optimization methods

Abstract (arXiv)

Activation verbalization methods such as Activation Oracle and Natural Language Autoencoders decode hidden representations of large language models into human-readable natural language. However, existing methods can produce incomplete or hallucinated descriptions, making their activation verbalizations difficult to trust and use reliably in practice. To this end, we introduce AVPO, a two-stage framework that first reconstructs source text from a hidden activation and then evaluates the resulting text with a separate frozen question-answering model, yielding an explicit and inspectable intermediate readout. We further optimize the inverter with direct preference optimization (DPO), using rewards that capture both semantic recoverability and lexical fidelity. Across six text families, AVPO improves gist- and detail-level information recovery over the strongest baseline by up to 17.1 and 9.3 percentage points, respectively. Crucially, the gains arise from preference optimization rather than fine-tuning on selected reconstructions alone, enabling compact cross-model inverters to surpass donor-matched question-conditioned verbalizers while improving both semantic recoverability and lexical fidelity. Moreover, out-of-distribution case study shows that AVPO better recovers high-level semantics while fabricating fewer details.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 30 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

21 of 30 references.

References · 21

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  8. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  9. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  10. 2024A StrongREJECT for Empty Jailbreakscited by 352
  11. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  12. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 189
9 more
  1. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 63
  2. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  3. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 41
  4. 2025Do Activation Verbalization Methods Convey Privileged Information?cited by 8
  5. 2025Rep2Text: Decoding Full Text from a Single LLM Token Representationcited by 7
  6. 2026Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanationcited by 2
  7. 2026Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanationscited by 2
  8. 2026Building Better Activation Oraclescited by 1
  9. 2026Retrieval is Enough: Training-Free Interpretability with a Tool-Using Agentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ActivationReasoning: Logical Reasoning in Latent Activation Spaces
  2. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning
  3. 2026Prompt Compression via Activation Aggregation
  4. 2026Measuring Maximum Activations in Open Large Language Models
  5. 2026UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering
  6. 2026Bypass Observation: A Conceptual Design of a Non-Intrusive Layer-Wise Semantic Extraction Architecture
  7. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?
  8. 2024100% Elimination of Hallucinations on RAGTruth for GPT-4 and GPT-3.5 Turbo