C-DeltaTheta: Circuit-Restricted Weight Arithmetic for Selective Refusal
Not on the map. Why?
Abstract (arXiv)
Modern deployments require LLMs to enforce safety policies at scale, yet many controls rely on inference-time interventions that add recurring compute cost and serving complexity. Activation steering is widely used, but it requires runtime hooks and scales cost with the number of generations; conditional variants improve selectivity by gating when steering is applied but still retain an inference-time control path. We ask whether selective refusal can be moved entirely offline: can a mechanistic understanding of category-specific refusal be distilled into a circuit-restricted weight update that deploys as a standard checkpoint? We propose C-Deltaθ Circuit Restricted Weight Arithmetic, which (i) localizes refusal-causal computation as a sparse circuit using EAP-IG and (ii) computes a constrained weight update DeltaθC supported only on that circuit (typically <5% of parameters). Applying DeltaθC yields a drop-in edited checkpoint with no inference-time hooks, shifting cost from per request intervention to a one-time offline update. We evaluate category-targeted selectivity and capability retention on refusal and utility benchmarks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
31 of 38 references.
References · 31
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
19 more
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
- 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
- 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
- 2024Programming Refusal with Conditional Activation Steeringcited by 191
- 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 144
- 2023Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Maskscited by 130
- 2024Localizing Task Information for Improved Model Merging and Compressioncited by 130
- 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 53
- 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
- 2024LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Mergingcited by 39
- 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
- 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
- 2025Steering Language Models with Weight Arithmeticcited by 19
Built on by
Not cited yet.