Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoE
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 2 more
Abstract (arXiv)
Safety alignment in multilingual models is uneven: a model that reliably refuses a harmful request in English will often comply with the same request in a lower-resource language. We trace this gap mechanistically in sarvam, an Indic-multilingual mixture-of-experts reasoning model, and find it is not a failure to detect harm. Harm is encoded as an internal direction that is nearly language-invariant in mid-network (English-vs-Indic cosine ≈0.9 at L11), and steering that direction upstream causally controls refusal. But the detection direction is orthogonal to the change that actually writes the refusal, which is late and assembled over the course of generation rather than read off in a single forward pass. We attribute the write to a specific, localizable circuit, a mixture-of-experts writer held in check by an attention opposer and price every way of intervening on it: damping the opposer is cheap and effective, amplifying the writer is a cost wall, and surgical edits to the responsible heads do nothing. The circuit's organization, and the gradient method that exposes it, recur in a second, unrelated MoE model, while the lever's strength is architecture-specific. The result is a cost-measured map of where a multilingual safety repair can land, and what it costs
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 4 of its 7 references that are in the map. First posted on arXiv 8 Aug 2026.
Builds on
4 of 7 references.
Closest ancestors · 1
Same direction, newest first
- 2025SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification20 Jun
Other references · 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Hard Negatives Reveal What Easy Negatives Hide: Cross-Lingual Harmfulness Representations Degrade with Resource Tier Under Hard Negatives
- 2026Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
- 2026BabelSteering: Multilingual Safety Alignment via English Steering Vectors
- 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
- 2026Decomposing Refusal Steering in Mixture-of-Experts Models
- 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
- 2026Low-Resource Safety Failures Are Action Failures, Not Representation Failures
- 2026TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety