# MentorCollab: Large-to-Small Inference-Time Mentorship for Concise Reasoning in Language Models

arXiv: https://arxiv.org/abs/2602.05307 · HTML: https://mlsanity.com/paper/2602.05307/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [Large Language Model Routing with Benchmark Datasets](https://mlsanity.com/paper/2309.15789/) (2023)
- [GraphRouter: A Graph-based Router for LLM Selections](https://mlsanity.com/paper/2410.03834/) (2024)
- [Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models](https://mlsanity.com/paper/2311.08692/) (2023)
- [Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning](https://mlsanity.com/paper/2310.03094/) (2023)
- [Language Model Cascades: Token-level uncertainty and beyond](https://mlsanity.com/paper/2404.10136/) (2024)
- [Learning to Decode Collaboratively with Multiple Language Models](https://mlsanity.com/paper/2403.03870/) (2024)
- [TensorOpera Router: A Multi-Model Router for Efficient LLM Inference](https://mlsanity.com/paper/2408.12320/) (2024)
- [Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization](https://mlsanity.com/paper/2404.11531/) (2024)
- [Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time](https://mlsanity.com/paper/2504.12329/) (2025)
- [CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing](https://mlsanity.com/paper/2502.01976/) (2025)
- [R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing](https://mlsanity.com/paper/2505.21600/) (2025)
- [Nudging: Inference-time Alignment of LLMs via Guided Decoding](https://mlsanity.com/paper/2410.09300/) (2024)
- [Do Cognitively Interpretable Reasoning Traces Improve LLM Performance?](https://mlsanity.com/paper/2508.16695/) (2025)
- [DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference](https://mlsanity.com/paper/2510.19669/) (2025)
- [Don't Throw Away Your Pretrained Model](https://mlsanity.com/paper/2510.09913/) (2025)
- [Speculate, then Collaborate: Fusing Knowledge of Language Models during Decoding](https://mlsanity.com/paper/2502.08020/) (2025)

## Built on by (most cited first, 2 of 2)

- [Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning](https://mlsanity.com/paper/2606.01682/) (2026)
- [Offline Guidance, Online Reasoning: Reusing LLM Feedback for Small Language Models](https://mlsanity.com/paper/2609.39346/) (2026)

## Abstract (arXiv)

Large reasoning models (LRMs) have demonstrated impressive reasoning capabilities, but their solutions are often verbose and computationally expensive, and taxing for users to read. In contrast, small language models (SLMs) produce concise outputs with lower inference costs, yet they frequently struggle on challenging multi-step reasoning tasks. Existing inference-time collaboration methods attempt to bridge this gap through imitation, encouraging SLMs to follow the reasoning process of LRMs. However, the student often inherits the mentor's overthinking, producing long and reflective reasoning chains while still falling short in accuracy. We propose MentorCollab, a collaboration method based on mentorship: the SLM remains the primary generator and consults the LRM only when additional reasoning support is needed. At sparsely sampled token positions, we probe for divergence between the two models and use a lightweight verifier to decide whether the SLM should follow a short lookahead segment from its mentor or continue on its own. Across 15 SLM-LRM pairs and 3 domains, our method achieves an average gain of 3.0%, with improvements of up to 8.0% in 12 settings. The resulting traces remain shorter than the mentor's, using only a small fraction of its tokens. These results demonstrate that selective, verified mentorship can boost reasoning accuracy while preserving concise generation.