# R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing

arXiv: https://arxiv.org/abs/2505.21600 · HTML: https://mlsanity.com/paper/2505.21600/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 37 of its 62 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 37)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention](https://mlsanity.com/paper/2502.11089/) (2025)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [Chain of Draft: Thinking Faster by Writing Less](https://mlsanity.com/paper/2502.18600/) (2025)
- [Token-Budget-Aware LLM Reasoning](https://mlsanity.com/paper/2412.18547/) (2024)
- [DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data](https://mlsanity.com/paper/2405.14333/) (2024)
- [A Survey on Efficient Inference for Large Language Models](https://mlsanity.com/paper/2404.14294/) (2024)
- [MoBA: Mixture of Block Attention for Long-Context LLMs](https://mlsanity.com/paper/2502.13189/) (2025)
- [DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search](https://mlsanity.com/paper/2408.08152/) (2024)
- [Are We Done with MMLU?](https://mlsanity.com/paper/2406.04127/) (2024)
- [Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs](https://mlsanity.com/paper/2501.18585/) (2025)
- [Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models](https://mlsanity.com/paper/2311.08692/) (2023)
- [A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond](https://mlsanity.com/paper/2503.21614/) (2025)
- [Reward-Guided Speculative Decoding for Efficient LLM Reasoning](https://mlsanity.com/paper/2501.19324/) (2025)
- [Learning Harmonized Representations for Speculative Sampling](https://mlsanity.com/paper/2408.15766/) (2024)
- [SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training](https://mlsanity.com/paper/2505.11594/) (2025)
- [MixLLM: Dynamic Routing in Mixed Large Language Models](https://mlsanity.com/paper/2502.18482/) (2025)

## Built on by (most cited first, 29 of 29)

- [Cache-to-Cache: Direct Semantic Communication Between Large Language Models](https://mlsanity.com/paper/2510.03215/) (2025)
- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models](https://mlsanity.com/paper/2506.16054/) (2025)
- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://mlsanity.com/paper/2603.04445/) (2026)
- [Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2507.09662/) (2025)
- [Semantic Energy: Detecting LLM Hallucination Beyond Entropy](https://mlsanity.com/paper/2508.14496/) (2025)
- [GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts](https://mlsanity.com/paper/2601.05110/) (2026)
- [Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning](https://mlsanity.com/paper/2511.08577/) (2025)
- [Token-Level LLM Collaboration via FusionRoute](https://mlsanity.com/paper/2601.05106/) (2026)
- [RelayLLM: Efficient Reasoning via Collaborative Decoding](https://mlsanity.com/paper/2601.05167/) (2026)
- [A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthiness](https://mlsanity.com/paper/2510.13890/) (2025)
- [FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning](https://mlsanity.com/paper/2601.19001/) (2026)
- [Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers](https://mlsanity.com/paper/2603.26380/) (2026)
- [Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization](https://mlsanity.com/paper/2608.16072/) (2026)
- [Select to Think: Unlocking SLM Potential with Local Sufficiency](https://mlsanity.com/paper/2604.26940/) (2026)
- [MentorCollab: Large-to-Small Inference-Time Mentorship for Concise Reasoning in Language Models](https://mlsanity.com/paper/2602.05307/) (2026)
- [Rethinking Model Efficiency: Multi-Agent Inference with Large Models](https://mlsanity.com/paper/2604.04929/) (2026)
- [Dissecting Failure Dynamics in Large Language Model Reasoning](https://mlsanity.com/paper/2604.14528/) (2026)
- [TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification](https://mlsanity.com/paper/2601.23180/) (2026)
- [RelayGen: Intra-Generation Model Switching for Efficient Reasoning](https://mlsanity.com/paper/2602.06454/) (2026)
- [RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning](https://mlsanity.com/paper/2604.16694/) (2026)
- [UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling](https://mlsanity.com/paper/2605.30898/) (2026)
- [FutureBridge: Token Selection Beyond Local Preference in Collaborative Decoding](https://mlsanity.com/paper/2608.06819/) (2026)
- [Improving Test-Time Scaling with Adaptive Looped Transformers](https://mlsanity.com/paper/2609.35748/) (2026)
- [Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference](https://mlsanity.com/paper/2604.13634/) (2026)
- [Compute Where it Counts: Self Optimizing Language Models](https://mlsanity.com/paper/2605.10875/) (2026)
- [Before Thinking, Learn to Decide: Proactive Routing for Efficient Visual Reasoning](https://mlsanity.com/paper/2606.30217/) (2026)
- [What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute](https://mlsanity.com/paper/2610.02491/) (2026)
- [TokenRouter: Efficient Serving System for Token-Level LLM Routing](https://mlsanity.com/paper/2610.12242/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) achieve impressive reasoning capabilities at the cost of substantial inference overhead, posing substantial deployment challenges. Although distilled Small Language Models (SLMs) significantly enhance efficiency, their performance suffers as they fail to follow LLMs' reasoning paths. Luckily, we reveal that only a small fraction of tokens genuinely diverge reasoning paths between LLMs and SLMs. Most generated tokens are either identical or exhibit neutral differences, such as minor variations in abbreviations or expressions. Leveraging this insight, we introduce **Roads to Rome (R2R)**, a neural token routing method that selectively utilizes LLMs only for these critical, path-divergent tokens, while leaving the majority of token generation to the SLM. We also develop an automatic data generation pipeline that identifies divergent tokens and generates token-level routing labels to train the lightweight router. We apply R2R to combine R1-1.5B and R1-32B models from the DeepSeek family, and evaluate on challenging math, coding, and QA benchmarks. With an average activated parameter size of 5.6B, R2R surpasses the average accuracy of R1-7B by 1.6x, outperforming even the R1-14B model. Compared to R1-32B, it delivers a 2.8x wall-clock speedup with comparable performance, advancing the Pareto frontier of test-time scaling efficiency. Our code is available at https://github.com/thu-nics/R2R.