mlsanityData to 14 Oct 2026

SafeNeuron: Neuron-Level Safety Alignment for Large Language Models

arXiv 2602.12158 ↗First posted 12 Feb 2026cs.LGcited by 10

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Large language models (LLMs) and multimodal LLMs are typically safety-aligned before release to prevent harmful content generation. However, recent studies show that safety behaviors are concentrated in a small subset of parameters, making alignment brittle and easily bypassed through neuron-level attacks. Moreover, most existing alignment methods operate at the behavioral level, offering limited control over the model's internal safety mechanisms. In this work, we propose SafeNeuron, a neuron-level safety alignment framework that improves robustness by redistributing safety representations across the network. SafeNeuron first identifies safety-related neurons, then freezes these neurons during preference optimization to prevent reliance on sparse safety pathways and force the model to construct redundant safety representations. Extensive experiments across models and modalities demonstrate that SafeNeuron significantly improves robustness against neuron pruning attacks, reduces the risk of open-source models being repurposed as red-team generators, and preserves general capabilities. Furthermore, our layer-wise analysis reveals that safety behaviors are governed by stable and shared internal representations. Overall, SafeNeuron provides an interpretable and robust perspective for model alignment.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 57 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

47 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models22 May
  2. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model17 Jun
  3. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  4. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2025Qwen2.5-VL Technical Reportcited by 5,587
  9. 2013Intriguing properties of neural networkscited by 5,155
  10. 2024Qwen2.5 Technical Reportcited by 4,942
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
31 more
  1. 2023Qwen Technical Reportcited by 3,344
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  7. 2025Kimi K2: Open Agentic Intelligencecited by 567
  8. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  9. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  10. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  11. 2024A StrongREJECT for Empty Jailbreakscited by 385
  12. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  13. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  14. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  15. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  16. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  17. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  18. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 112
  19. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 110
  20. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 84
  21. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 39
  22. 2024BlockPruner: Fine-grained Pruning for Large Language Modelscited by 36
  23. 2025Security Concerns for Large Language Models: A Surveycited by 29
  24. 2025NeuroStrike: Neuron-Level Attacks on Aligned LLMscited by 22
  25. 2025Principled Multimodal Representation Learningcited by 19
  26. 2025Safety Alignment via Constrained Knowledge Unlearningcited by 14
  27. 2024Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspectivecited by 13
  28. 2025Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Modelscited by 9
  29. 2025Fewer Weights, More Problems: A Practical Attack on LLM Pruningcited by 7
  30. 2025NeuRel-Attack: Neuron Relearning for Safety Disalignment in Large Language Modelscited by 6
  31. 2025NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Multilingual Safety Alignment Via Sparse Weight Editingcited by 6
  2. 2026A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Modelscited by 3
  3. 2026NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistributioncited by 2
  4. 2026Targeted Neuron Modulation via Contrastive Pair Searchcited by 1
  5. 2026Targeted Interpretable Safety Neuron Enhancement for Multilingual Vision-Language Large Modelscited by 0
  6. 2026Mask2Shield: Strengthening LLM Safety against Neuron-Pruning Attackscited by 0
  7. 2026One Anchor for All: Unified Multilingual and Multimodal Safety Alignment for LVLMscited by 0
  8. 2026No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attackscited by 0
  9. 2026Moving the Safety Barrier: Dynamic Routing Adaptive Alignment Against White-Box Attackscited by 0
  10. 2026SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
  2. 2026NeST: Neuron Selective Tuning for LLM Safety
  3. 2026SafeNexus: Discovering and Steering Modality-Universal Safety Neurons in MLLMs
  4. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning
  5. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  6. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
  7. 2026Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
  8. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications