mlsanityData to 7 Oct 2026

SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models

arXiv 2610.09600 ↗First posted 7 Oct 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety interpretability advances the study of Large Language Model (LLM) alignment from behavioral constraints driven by data or algorithms towards a deeper understanding of internal mechanisms. However, existing works have focused primarily on safety-related representations, attention heads, or neurons after alignment, while largely overlooking the safety mechanisms in pretrained-only models and their evolution across alignment checkpoints. To address this, we propose SafeEvo, an interpretability framework from the circuit (sparse subgraphs of an LLM) perspective. SafeEvo first applies an optimization-based extraction algorithm to identify weak refusal circuits in pretrained base LLMs that can independently express refusal behavior. Causally ablating these circuits completely eliminates the base model's refusal of harmful inputs. SafeEvo then traces the evolution of refusal circuits across successive alignment checkpoints and finds that their structures change progressively, suggesting that the alignment tax may result from refusal-circuit updates affecting utility-related parameters. To validate this, SafeEvo introduces Safety Circuit Alignment (SCA), which confines safety updates to the refusal circuits. Experiments across three LLMs and two alignment algorithms show that, on average, SCA outperforms vanilla alignment in three aspects: (1) stronger alignment, lowering harmfulness score by 63.21%; (2) less over-refusal, yielding a 58.44% decrease in refusal rates for benign queries; and (3) better utility, retaining 99.58% of the original model capabilities.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 64 references that are in the map. First posted on arXiv 7 Oct 2026.

Builds on

24 of 64 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026NeST: Neuron Selective Tuning for LLM Safety18 Feb
  2. 2026SafeNeuron: Neuron-Level Safety Alignment for Large Language Models12 Feb
  3. 2026What Matters For Safety Alignment?7 Jan
  4. 2025Improving LLM Safety Alignment with Dual-Objective Optimization5 Mar
  5. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning17 Dec
  6. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons20 Jun
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2023Mistral 7Bcited by 3,065
  5. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  6. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  8. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 280
  9. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  10. 2026Kimi K3: Open Frontier Intelligencecited by 99
  11. 2026Mechanistic Interpretability for Large Language Model Alignment: Progress, Challenges, and Future Directionscited by 8
  12. 2025Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Modelscited by 7
5 more
  1. 2025Interpretable LLM Guardrails via Sparse Representation Steeringcited by 5
  2. 2026SafeSeek: Universal Attribution of Safety Circuits in Language Modelscited by 4
  3. 2026A Survey of Large Audio Language Models: Generalization, Trustworthiness, and Outlookcited by 4
  4. 2026SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systemscited by 2
  5. 2026Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models
  2. 2026When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning
  3. 2026Fail-Closed Alignment for Large Language Models
  4. 2026The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs
  5. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
  6. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
  7. 2025Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMs
  8. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models