mlsanityData to 7 Oct 2026

BFS-PO: Best-First Search for Large Reasoning Models

arXiv 2602.14917 ↗First posted 16 Feb 2026cs.CL cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Reasoning Models (LRMs) such as OpenAI o1 and DeepSeek-R1 have shown excellent performance in reasoning tasks using long reasoning chains. However, this has also led to a significant increase of computational costs and the generation of verbose output, a phenomenon known as overthinking. The tendency to overthinking is often exacerbated by Reinforcement Learning (RL) algorithms such as GRPO/DAPO. In this paper, we propose BFS-PO, an RL algorithm which alleviates this problem using a Best-First Search exploration strategy. Specifically, BFS-PO looks for the shortest correct answer using a backtracking mechanism based on maximum entropy nodes. By generating progressively shorter responses during training, BFS-PO learns to produce concise reasoning chains. Using different benchmarks and base LRMs, we show that BFS-PO can simultaneously increase the LRM accuracy and shorten its answers.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 63 of its 68 references that are in the map. First posted on arXiv 16 Feb 2026.

Builds on

63 of 68 references.

Closest ancestors · 32

Same direction, newest first

  1. 2025Controlling Thinking Speed in Reasoning Models4 Jul
  2. 2025A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings30 May
  3. 2025LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling25 May
  4. 2025First Finish Search: Efficient Test-Time Scaling in Large Language Models23 May
  5. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficient23 May
  6. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning23 May
  7. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains22 May
  8. 2025Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning22 May
  9. 2025ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancy21 May
  10. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuning20 May
  11. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning20 May
  12. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning8 May
20 more
  1. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs30 Apr
  2. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning30 Apr
  3. 2025Concise Reasoning via Reinforcement Learning7 Apr
  4. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr
  5. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  6. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning25 Feb
  7. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb
  8. 2025The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer21 Feb
  9. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  10. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  11. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  12. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb
  13. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning5 Feb
  14. 2025Latent Thought Models with Variational Bayes Inference-Time Computation3 Feb
  15. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  16. 2024Efficiently Scaling LLM Reasoning with Certaindex30 Dec
  17. 2024Token-Budget-Aware LLM Reasoning24 Dec
  18. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness16 Dec
  19. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  20. 2023Large Language Model Guided Tree-of-Thought15 May

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2024Qwen2 Technical Reportcited by 2,076
  11. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
19 more
  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  4. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  5. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  6. 2025Training Language Models to Reason Efficientlycited by 242
  7. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 225
  8. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  9. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 165
  10. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  11. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 86
  12. 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Searchcited by 75
  13. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  14. 2025LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinkingcited by 34
  15. 2025Soft Tokens, Hard Truthscited by 31
  16. 2025Implicit Reasoning in Transformers is Reasoning through Shortcutscited by 14
  17. 2024Anchor-based Large Language Modelscited by 12
  18. 2024Can We Afford The Perfect Prompt? Balancing Cost and Accuracy with the Economical Prompting Indexcited by 4
  19. 2024Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteriacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learningcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  4. 2025How Far Are We from Optimal Reasoning Efficiency?
  5. 2025QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
  6. 2025DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
  7. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
  8. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models