# Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits

arXiv: https://arxiv.org/abs/2511.07482 · HTML: https://mlsanity.com/paper/2511.07482/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 16 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 14 of 14)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [LLM-Pruner: On the Structural Pruning of Large Language Models](https://mlsanity.com/paper/2305.11627/) (2023)
- [Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications](https://mlsanity.com/paper/2402.05162/) (2024)
- [WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models](https://mlsanity.com/paper/2406.18510/) (2024)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022)
- [SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation](https://mlsanity.com/paper/2501.01765/) (2025)
- [NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning](https://mlsanity.com/paper/2412.12497/) (2024)
- [Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs](https://mlsanity.com/paper/2406.16797/) (2024)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Understanding Layer Significance in LLM Alignment](https://mlsanity.com/paper/2410.17875/) (2024)

## Built on by (most cited first, 4 of 4)

- [Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?](https://mlsanity.com/paper/2605.28860/) (2026)
- [POP: Online Structural Pruning Enables Efficient Inference of Large Foundation Models](https://mlsanity.com/paper/2602.06822/) (2026)
- [LLM Scheming Inversely Scales with Pretraining Language Coverage](https://mlsanity.com/paper/2607.24769/) (2026)
- [Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates](https://mlsanity.com/paper/2607.28959/) (2026)

## Abstract (arXiv)

Large Language Models require substantial computational resources for inference, posing deployment challenges. While dynamic pruning offers superior efficiency over static methods through adaptive circuit selection, it exacerbates alignment degradation by retaining only input-dependent safety-critical circuit preservation across diverse inputs. As a result, addressing these heightened alignment vulnerabilities remains critical. We introduce Alignment-Aware Probe Pruning (AAPP), a dynamic structured pruning method that adaptively preserves alignment-relevant circuits during inference, building upon Probe Pruning. Experiments on LLaMA 2-7B, Qwen2.5-14B-Instruct, and Gemma-3-12B-IT show AAPP improves refusal rates by 50% at matched compute, enabling efficient yet safety-preserving LLM deployment.