mlsanityData to 7 Oct 2026

Does the Same Token Mean the Same State? MoE Routing as Signal for Reasoning Control

arXiv 2606.22798 ↗First posted 22 Jun 2026cs.CLcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

In sparse Mixture-of-Experts language models, does the same token id imply the same router state and the same experts producing it? Holding the emitted token id fixed at repeated anchors, we find it does not: the experts that produce it still separate task context, trajectory history, and reasoning-effort mode. This residual structure supports test-time control: near boundary anchors (the final-response transition) and delimiter anchors (which open the answer, e.g. texttttextbackslash boxed or code fences), routing neighborhoods already align with final-answer basins at a marker-only readout and strongest when the routing is read at the answer opening. We operationalize this as RAD (Routing Agreement Decoding), an answer-string-free multi-rollout selector: it locates a fixed anchor, represents each rollout by its anchor-window MoE routing states, and returns the densest Weighted-Jaccard K-NN route-basin center, without parsing, normalizing, executing, or voting over answer strings. Across 10 sparse-MoE configurations (gpt-oss, Qwen3-MoE) and 6 datasets spanning math, GPQA, and code, RAD is on par with Majority where string voting is well-posed, with small positive paired deltas (RAD 73.9 / RAD+DC 74.2 vs. Majority 73.6). Like majority voting, RAD is not a verifier: a dense wrong basin can still win. Its value is the interface: the same selector gives direct pass@1 on code, where exact-string voting is ill-defined, and the same routing-density principle, re-anchored to the agentic boundary, improves best-of-16 patch selection on SWE-bench Verified over random, where patches have no answer string to vote on.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 14 of its 26 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

14 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026NEX: Neuron Explore-Exploit Scoring for Label-Free Chain-of-Thought Selection and Model Ranking5 Feb
  2. 2025Do LLMs Signal When They're Right? Evidence from Neuron Agreement30 Oct
  3. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification7 Apr

Other references · 11

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2024Mixtral of Expertscited by 1,785
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2025Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metriccited by 10
  8. 2026The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertisecited by 9
  9. 2025Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanismscited by 6
  10. 2026In harmony with gpt-osscited by 1
  11. 2026Layer-wise MoE Routing Locality under Shared-Prefix Code Generation: Token-Identity Decomposition and Compile-Equivalent Fork Redundancycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
  2. 2026Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing
  3. 2026Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality
  4. 2026MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale
  5. 2026Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
  6. 2025R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing
  7. 2026Is MoE Routing a Huffman Code? Discovering the Frequency-Diversity Law in Chain-of-Thought
  8. 2026On the Token Value Inequality in Efficient Reasoning