# Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling

arXiv: https://arxiv.org/abs/2601.21684 · HTML: https://mlsanity.com/paper/2601.21684/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 27 of its 61 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 27 of 27)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025)
- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [Scalable Best-of-N Selection for Large Language Models via Self-Certainty](https://mlsanity.com/paper/2502.18581/) (2025)
- [Phi-4-reasoning Technical Report](https://mlsanity.com/paper/2504.21318/) (2025)
- [ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization](https://mlsanity.com/paper/2509.13313/) (2025)
- [Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling](https://mlsanity.com/paper/2502.06703/) (2025)
- [ACON: Optimizing Context Compression for Long-horizon LLM Agents](https://mlsanity.com/paper/2510.00615/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding](https://mlsanity.com/paper/2309.15028/) (2023)
- [Training-Free Group Relative Policy Optimization](https://mlsanity.com/paper/2510.08191/) (2025)
- [Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math](https://mlsanity.com/paper/2510.13744/) (2025)
- [Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learning](https://mlsanity.com/paper/2512.01282/) (2025)
- [PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning](https://mlsanity.com/paper/2601.05593/) (2026)
- [Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling](https://mlsanity.com/paper/2506.15707/) (2025)
- [Batch-of-Thought: Cross-Instance Learning for Enhanced LLM Reasoning](https://mlsanity.com/paper/2601.02950/) (2026)

## Built on by (most cited first, 9 of 9)

- [Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering](https://mlsanity.com/paper/2604.08224/) (2026)
- [LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling](https://mlsanity.com/paper/2605.08083/) (2026)
- [OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Search](https://mlsanity.com/paper/2604.03675/) (2026)
- [Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scaling](https://mlsanity.com/paper/2605.27030/) (2026)
- [Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling](https://mlsanity.com/paper/2606.03102/) (2026)
- [TMAS: Scaling Test-Time Compute via Multi-Agent Synergy](https://mlsanity.com/paper/2605.10344/) (2026)
- [Hyper-KGGen: A Skill-Driven Knowledge Extractor for High-Quality Knowledge Hypergraph Generation](https://mlsanity.com/paper/2602.19543/) (2026)
- [Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs](https://mlsanity.com/paper/2610.00047/) (2026)
- [From Pareto to Preference: Personalized Test-Time Scaling via Amortized Agentic Policy Discovery](https://mlsanity.com/paper/2610.09684/) (2026)

## Abstract (arXiv)

Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration of the solution space. However, existing search strategies typically treat rollouts as disposable samples, where valuable intermediate insights are effectively discarded after each trial. This wasted rollout-level experience leads to substantial computational redundancy, as models repeatedly re-derive discovered conclusions and revisit known dead ends across extensive attempts. To bridge this gap, we propose Recycling Search Experience (RSE), a self-guided, training-free strategy that turns test-time search from a series of isolated trials into a cumulative, experience-guided process. By actively distilling raw trajectories into a shared experience bank, RSE enables positive recycling of intermediate conclusions to shortcut redundant derivations and negative recycling of failure patterns to prune encountered dead ends. Theoretically, we provide an analysis that formalizes the efficiency gains of RSE over independent sampling in solving complex reasoning tasks. Empirically, extensive experiments on HMMT24, HMMT25, IMO-Bench, and HLE show that RSE consistently outperforms strong baselines under comparable computational budgets, establishing a strong compute-efficiency frontier for test-time scaling.