mlsanityData to 7 Oct 2026

Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoE

arXiv 2608.08032 ↗First posted 8 Aug 2026cs.AI cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 2 more
Abstract (arXiv)

Safety alignment in multilingual models is uneven: a model that reliably refuses a harmful request in English will often comply with the same request in a lower-resource language. We trace this gap mechanistically in sarvam, an Indic-multilingual mixture-of-experts reasoning model, and find it is not a failure to detect harm. Harm is encoded as an internal direction that is nearly language-invariant in mid-network (English-vs-Indic cosine ≈0.9 at L11), and steering that direction upstream causally controls refusal. But the detection direction is orthogonal to the change that actually writes the refusal, which is late and assembled over the course of generation rather than read off in a single forward pass. We attribute the write to a specific, localizable circuit, a mixture-of-experts writer held in check by an attention opposer and price every way of intervening on it: damping the opposer is cheap and effective, amplifying the writer is a cost wall, and surgical edits to the responsible heads do nothing. The circuit's organization, and the gradient method that exposes it, recur in a second, unrelated MoE model, while the lever's strength is architecture-specific. The result is a cost-measured map of where a multilingual safety repair can land, and what it costs

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 4 of its 7 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

4 of 7 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification20 Jun

Other references · 3

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2025Refusal Direction is Universal Across Safety-Aligned Languagescited by 37
  3. 2025TwinBreak: Jailbreaking LLM Security Alignments based on Twin Promptscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Hard Negatives Reveal What Easy Negatives Hide: Cross-Lingual Harmfulness Representations Degrade with Resource Tier Under Hard Negatives
  2. 2026Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
  3. 2026BabelSteering: Multilingual Safety Alignment via English Steering Vectors
  4. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
  5. 2026Decomposing Refusal Steering in Mixture-of-Experts Models
  6. 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
  7. 2026Low-Resource Safety Failures Are Action Failures, Not Representation Failures
  8. 2026TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety