mlsanityData to 7 Oct 2026

C-DeltaTheta: Circuit-Restricted Weight Arithmetic for Selective Refusal

arXiv 2602.04521 ↗First posted 4 Feb 2026cs.CL cs.ETcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern deployments require LLMs to enforce safety policies at scale, yet many controls rely on inference-time interventions that add recurring compute cost and serving complexity. Activation steering is widely used, but it requires runtime hooks and scales cost with the number of generations; conditional variants improve selectivity by gating when steering is applied but still retain an inference-time control path. We ask whether selective refusal can be moved entirely offline: can a mechanistic understanding of category-specific refusal be distilled into a circuit-restricted weight update that deploys as a standard checkpoint? We propose C-Deltaθ Circuit Restricted Weight Arithmetic, which (i) localizes refusal-causal computation as a sparse circuit using EAP-IG and (ii) computes a constrained weight update DeltaθC supported only on that circuit (typically <5% of parameters). Applying DeltaθC yields a drop-in edited checkpoint with no inference-time hooks, shifting cost from per request intervention to a one-time offline update. We evaluate category-targeted selectivity and capability retention on refusal and utility benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

31 of 38 references.

References · 31

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  10. 2022Editing Models with Task Arithmeticcited by 1,076
  11. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  12. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
19 more
  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2023Steering Language Models With Activation Engineeringcited by 824
  3. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  4. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  5. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  6. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  7. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  8. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  9. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  10. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  11. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 144
  12. 2023Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Maskscited by 130
  13. 2024Localizing Task Information for Improved Model Merging and Compressioncited by 130
  14. 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 53
  15. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  16. 2024LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Mergingcited by 39
  17. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  18. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  19. 2025Steering Language Models with Weight Arithmeticcited by 19

Built on by

Not cited yet.