Bridge Routing Heads: Where Multilingual Multi-hop Reasoning Lives in LLMs
On the map: LLM reasoning → Reasoning benchmarks and evaluation also fits: Prompting and chain-of-thought reasoning
Abstract (arXiv)
Multilingual LLMs answer the same multi-hop reasoning question across languages, but we lack a mechanistic account of whether they share an internal circuit. We identify Bridge Routing Heads (BRH) in two large multilingual LLMs through a three-stage pipeline. The resulting language-specific head sets exhibit near-complete mutual exclusivity across the five languages, with a mean Jaccard similarity of only 0.017 for Llama 3.1 70B and 0.057 for Qwen 2.5 72B, revealing language-idiosyncratic circuits. Ablating general BRH increases two-hop Negative Log-Likelihood (NLL) by 39-89x the random-head baseline, providing direct causal evidence of their role. Amplifying these heads in a failing target-language pass rescues up to 51.7% of cross-lingual failures, with no training. The two models share this dual-circuit pattern but allocate heads differently: Llama concentrates chaining in a large general pool, while Qwen leans on larger language-specific pools. Together these results show that activation-level intervention alone can recover correct answers from cross-lingual reasoning failures.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 2 of its 24 references that are in the map. First posted on arXiv 7 Oct 2026.
Builds on
2 of 24 references.
References · 2
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2024Qwen2.5 Technical Reportcited by 4,713
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models
- 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
- 2025A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
- 2024Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?
- 2025Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
- 2024Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge
- 2025When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
- 2025Multilingual Routing in Mixture-of-Experts