# Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking

arXiv: https://arxiv.org/abs/2601.18146 · HTML: https://mlsanity.com/paper/2601.18146/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 15 of 15)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning](https://mlsanity.com/paper/2503.04697/) (2025)
- [ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning](https://mlsanity.com/paper/2504.01296/) (2025)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation](https://mlsanity.com/paper/2503.22675/) (2025)
- [ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability](https://mlsanity.com/paper/2508.07050/) (2025)
- [ARM: Adaptive Reasoning Model](https://mlsanity.com/paper/2505.20258/) (2025)
- [Top-Down Partitioning for Efficient List-Wise Ranking](https://mlsanity.com/paper/2405.14589/) (2024)
- [Check-Eval: A Checklist-based Approach for Evaluating Text Quality](https://mlsanity.com/paper/2407.14467/) (2024)
- [Toward Adaptive Reasoning in Large Language Models with Thought Rollback](https://mlsanity.com/paper/2412.19707/) (2024)
- [AdapThink: Adaptive Thinking Preferences for Reasoning Language Model](https://mlsanity.com/paper/2506.18237/) (2025)
- [KGBridge: Knowledge-Guided Prompt Learning for Non-overlapping Cross-Domain Recommendation](https://mlsanity.com/paper/2511.02181/) (2025)

## Built on by (most cited first, 4 of 4)

- [R2-Router: A New Paradigm for LLM Routing with Reasoning](https://mlsanity.com/paper/2602.02823/) (2026)
- [SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks](https://mlsanity.com/paper/2607.00053/) (2026)
- [Post-Reasoning: Improving the Performance of Non-Thinking Models at No Cost](https://mlsanity.com/paper/2605.06165/) (2026)
- [Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning](https://mlsanity.com/paper/2606.06835/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly applied to ranking tasks in retrieval and recommendation. Although reasoning prompting can enhance ranking utility, our preliminary exploration reveals that its benefits are inconsistent and come at a substantial computational cost, suggesting that when to reason is as crucial as how to reason. To address this issue, we propose a reasoning routing framework that employs a lightweight, plug-and-play router head to decide whether to use direct inference (Non-Think) or reasoning (Think) for each instance before generation. The router head relies solely on pre-generation signals: i) compact ranking-aware features (e.g., candidate dispersion) and ii) model-aware difficulty signals derived from a diagnostic checklist reflecting the model's estimated need for reasoning. By leveraging these features before generation, the router outputs a controllable token that determines whether to apply the Think mode. Furthermore, the router can adaptively select its operating policy along the validation Pareto frontier during deployment, enabling dynamic allocation of computational resources toward instances most likely to benefit from Think under varying system constraints. Experiments on three public ranking datasets with different scales of open-source LLMs show consistent improvements in ranking utility with reduced token consumption (e.g., +6.3% NDCG@10 with -49.5% tokens on MovieLens with Qwen3-4B), demonstrating reasoning routing as a practical solution to the accuracy-efficiency trade-off.