mlsanityData to 7 Oct 2026

RF-Agent: Automated Reward Function Design via Language Agent Tree Search

arXiv 2602.23876 ↗First posted 27 Feb 2026cs.AI cs.LGcited by 3

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Designing efficient reward functions for low-level control tasks is a challenging problem. Recent research aims to reduce reliance on expert experience by using Large Language Models (LLMs) with task information to generate dense reward functions. These methods typically rely on training results as feedback, iteratively generating new reward functions with greedy or evolutionary algorithms. However, they suffer from poor utilization of historical feedback and inefficient search, resulting in limited improvements in complex control tasks. To address this challenge, we propose RF-Agent, a framework that treats LLMs as language agents and frames reward function design as a sequential decision-making process, enhancing optimization through better contextual reasoning. RF-Agent integrates Monte Carlo Tree Search (MCTS) to manage the reward design and optimization process, leveraging the multi-stage contextual reasoning ability of LLMs. This approach better utilizes historical information and improves search efficiency to identify promising reward functions. Outstanding experimental results in 17 diverse low-level control tasks demonstrate the effectiveness of our method. The source code is available at https://github.com/deng-ai-lab/RF-Agent.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 49 of its 64 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

49 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May

Other references · 48

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2022Emergent Abilities of Large Language Modelscited by 2,483
36 more
  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  8. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  9. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  10. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  11. 2023A Comprehensive Overview of Large Language Modelscited by 639
  12. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  13. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  14. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  15. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  16. 2017Deep Reinforcement Learning: An Overviewcited by 327
  17. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  18. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  19. 2023Reward Design with Language Modelscited by 257
  20. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  21. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  22. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  23. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  24. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 137
  25. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  26. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  27. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  28. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  29. 2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 96
  30. 2022Advanced Skills by Learning Locomotion and Local Navigation End-to-Endcited by 39
  31. 2024True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learningcited by 36
  32. 2024REvolve: Reward Evolution with Large Language Models using Human Feedbackcited by 22
  33. 2025A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworkscited by 21
  34. 2024Learning Reward for Robot Skills Using Large Language Models via Self-Alignmentcited by 19
  35. 2024ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimizationcited by 6
  36. 2025Boosting Universal LLM Reward Design through Heuristic Reward Observation Space Evolutioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Executioncited by 3
  2. 2026Beyond Scripted Search: Sample-Efficient Reward Discovery via Agentic Black-box Optimizationcited by 0
  3. 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unifying Tree Search Algorithm and Reward Design for LLM Reasoning: A Survey
  2. 2026Reward-Driven LLM Agent Workflows: Synthesizing POMDP Routing and Self-Correction for Autonomous Decision-Making
  3. 2024Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement
  4. 2024On the Modeling Capabilities of Large Language Models for Sequential Decision Making
  5. 2025ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learning
  6. 2025Self-correcting Reward Shaping via Language Models for Reinforcement Learning Agents in Games
  7. 2026Enhanced LLM Reasoning by Optimizing Reward Functions with Search-Driven Reinforcement Learning
  8. 2025Scaling Autonomous Agents via Automatic Reward Modeling And Planning