mlsanityData to 14 Oct 2026

TGPR: Tree-Guided Policy Refinement for Robust Self-Debugging of LLMs

arXiv 2510.06878 ↗First posted 8 Oct 2025cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Iterative refinement has been a promising paradigm to enable large language models (LLMs) to resolve difficult reasoning and problem-solving tasks. One of the key challenges, however, is how to effectively search through the enormous search space of possible refinements. Existing methods typically fall back on predefined heuristics, which are troubled by the exploration-exploitation dilemma and cannot adapt based on past refinement outcomes. We introduce Tree-Guided Policy Refinement (TGPR), a novel framework that combines GRPO with a Thompson-Sampling-based tree search. TGPR explores both failed and successful refinement paths actively, with denser training trajectories and more adaptive policies. On HumanEval, MBPP, and APPS benchmarks, our method achieves up to +4.2 percentage points absolute improvement in pass@1 (on MBPP) and up to +12.51 percentage points absolute improvement in pass@10 (on APPS) compared to a competitive GRPO baseline. Apart from debugging code, TGPR focuses on a principled approach to combining learned policies with structured search methods, offering a general framework for enhancing iterative refinement and stateful reasoning in LLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 18 references that are in the map. First posted on arXiv 8 Oct 2025.

Builds on

15 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design15 Jan

Other references · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
  7. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  8. 2024Evaluating Language Models for Efficient Code Generationcited by 73
  9. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 51
  10. 2024Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedbackcited by 46
  11. 2025CodeContests+: High-Quality Test Case Generation for Competitive Programmingcited by 46
  12. 2023Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Studycited by 44
2 more
  1. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 40
  2. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ThinkTwice: Jointly Optimizing Large Language Models for Reasoning and Self-Refinement
  2. 2025Policy Guided Tree Search for Enhanced LLM Reasoning
  3. 2026Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution
  4. 2025From Implicit Exploration to Structured Reasoning: Leveraging Guideline and Refinement for LLMs
  5. 2026ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models
  6. 2025Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search
  7. 2026ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning
  8. 2025MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time