mlsanityData to 14 Oct 2026

Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability

arXiv 2504.04215 ↗First posted 5 Apr 2025cs.CL cs.AIcited by 7

On the map: Mechanistic interpretability → Steering and probing for LLM safety

Abstract (arXiv)

The rapid growth of large language models has spurred significant interest in model compression as a means to enhance their accessibility and practicality. While extensive research has explored model compression through the lens of safety, findings suggest that safety-aligned models often lose elements of trustworthiness post-compression. Simultaneously, the field of mechanistic interpretability has gained traction, with notable discoveries, such as the identification of a single direction in the residual stream mediating refusal behaviors across diverse model architectures. In this work, we investigate the safety of compressed models by examining the mechanisms of refusal, adopting a novel interpretability-driven perspective to evaluate model safety. Furthermore, leveraging insights from our interpretability analysis, we propose a lightweight, computationally efficient method to enhance the safety of compressed models without compromising their performance or utility.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability. Builds on 53 of its 76 references that are in the map. First posted on arXiv 5 Apr 2025.

Builds on

53 of 76 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
38 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  2. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  9. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  10. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  11. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  12. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  13. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  14. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  15. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  16. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  17. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  18. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  19. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  20. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
  21. 2022Language model compression with weighted low-rank factorizationcited by 195
  22. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
  23. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  24. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  25. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
  26. 2023Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimizationcited by 124
  27. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 108
  28. 2023LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximationcited by 104
  29. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 81
  30. 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 79
  31. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  32. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 57
  33. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 55
  34. 2023Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanismcited by 33
  35. 2024Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compressioncited by 33
  36. 2024Refusal in LLMs is an Affine Functioncited by 29
  37. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signalscited by 29
  38. 2022Can Model Compression Improve NLP Fairnesscited by 20

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantizationcited by 5
  2. 2026Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levelscited by 2
  3. 2026Quality Is Not a Safety Proxy Under Quantizationcited by 1
  4. 2026Which Decisions Low-Bit Quantization Breaks, and How to Predict Themcited by 1
  5. 2026Mechanistically Interpreting Compression in Vision-Language Modelscited by 0
  6. 2026When Personality Meets Quantization: A Layer-wise MBTI Analysis of Quantized LLMscited by 0
  7. 2026Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries
  2. 2025Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal
  3. 2025Understanding and Mitigating Over-refusal for Large Language Models via Representation Intervention
  4. 2024Steering Language Model Refusal with Sparse Autoencoders
  5. 2025Downsized and Compromised?: Assessing the Faithfulness of Model Compression
  6. 2026Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
  7. 2026Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
  8. 2026LLMs Can Unlearn Refusal with Only 1,000 Benign Samples