Detection Is Cheap, Routing Is Learned: Why Refusal-Based Alignment Evaluation Fails
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Current alignment evaluation mostly measures whether models encode dangerous concepts and whether they refuse harmful requests. Both miss the layer where alignment often operates: routing from concept detection to behavioral policy. We study political censorship in Chinese-origin language models as a natural experiment, using probes, surgical ablations, and behavioral tests across nine open-weight models from five labs. Three findings follow. First, probe accuracy alone is non-diagnostic: political probes, null controls, and permutation baselines can all reach 100%, so held-out category generalization is the informative test. Second, surgical ablation reveals lab-specific routing. Removing the political-sensitivity direction eliminates censorship and restores accurate factual output in most models tested, while one model confabulates because its architecture entangles factual knowledge with the censorship mechanism. Cross-model transfer fails, indicating that routing geometry is model- and lab-specific. Third, refusal is no longer the dominant censorship mechanism. Within one model family, hard refusal falls to zero while narrative steering rises to the maximum, making censorship invisible to refusal-only benchmarks. These results support a three-stage descriptive framework: detect, route, generate. Models often retain the relevant knowledge; alignment changes how that knowledge is expressed. Evaluations that audit only detection or refusal therefore miss the routing mechanism that most directly determines behavior.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 0 of its 1 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
0 of 1 references.
No references here yet.
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Modelscited by 3
- 2026Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applicationscited by 1
- 2026Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapterscited by 0
- 2026Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications17 May 2026
- 2026How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models6 Apr 2026
- 2026Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapters25 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026How China-Origin Vision-Language Models Move from Refusal to Reframing in State Alignment
- 2025Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek
- 2026How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense
- 2025Are LLMs Good Safety Agents or a Propaganda Engine?
- 2026Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreak
- 2025Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation
- 2025The Alignment Veto: How Safety Training Suppresses Cultural Knowledge in LLMs
- 2026We're Cooked! - Probing LLM Political Alignment Via Conflict-Framed Recipe Translation