A Study on Leveraging Search and Self-Feedback for Agent Reasoning
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
Recent works have demonstrated that incorporating search during inference can significantly improve reasoning capabilities of language agents. Some approaches may make use of the ground truth or rely on model's own generated feedback. The search algorithm uses this feedback to then produce values that will update its criterion for exploring and exploiting various reasoning paths. In this study, we investigate how search and model's self-feedback can be leveraged for reasoning tasks. First, we explore differences in ground-truth feedback and self-feedback during search for math reasoning. Second, we observe limitations in applying search techniques to more complex tasks like tool-calling and design domain-specific approaches to address these gaps. Our experiments reveal challenges related to generalization when solely relying on self-feedback during search. For search to work effectively, either access to the ground-truth is needed or feedback mechanisms need to be carefully designed for the specific task.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 13 of its 18 references that are in the map. First posted on arXiv 17 Feb 2025.
Builds on
13 of 18 references.
Closest ancestors · 3
Same direction, newest first
- 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents13 Aug
- 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models6 Oct
- 2023Reasoning with Language Model is Planning with World Model24 May
Other references · 10
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2025s1: Simple test-time scalingcited by 1,341
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
- 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
- 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
- 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 142
- 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 109
- 2025SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scalingcited by 54
- 2023Grounding Gaps in Language Model Generationscited by 47
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning
- 2026Search, Do not Guess: Teaching Small Language Models to Be Effective Search Agents
- 2025RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection
- 2026OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces
- 2026SelfSearch: Reward-Free Search for Self-Improving Agents
- 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
- 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation
- 2025Reinforcement Fine-Tuning for Reasoning towards Multi-Step Multi-Source Search in Large Language Models