mlsanityData to 7 Oct 2026

The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning

arXiv 2502.15214 ↗First posted 21 Feb 2025cs.LG cs.AI cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) has shown impressive results in sequential decision-making tasks. Meanwhile, Large Language Models (LLMs) and Vision-Language Models (VLMs) have emerged, exhibiting impressive capabilities in multimodal understanding and reasoning. These advances have led to a surge of research integrating LLMs and VLMs into RL. In this survey, we review representative works in which LLMs and VLMs are used to overcome key challenges in RL, such as lack of prior knowledge, long-horizon planning, and reward design. We present a taxonomy that categorizes these LLM/VLM-assisted RL approaches into three roles: agent, planner, and reward. We conclude by exploring open problems, including grounding, bias mitigation, improved representations, and action advice. By consolidating existing research and identifying future directions, this survey establishes a framework for integrating LLMs and VLMs into RL, advancing approaches that unify natural language and visual understanding with sequential decision-making.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

42 of 54 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  9. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  11. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  12. 2023Reward Design with Language Modelscited by 257
30 more
  1. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  2. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  3. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  4. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  5. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  6. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  7. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
  8. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  9. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 110
  10. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  11. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  12. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 61
  13. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 51
  14. 2024AGILE: A Novel Reinforcement Learning Framework of LLM Agentscited by 45
  15. 2023Large Language Models Are Semi-Parametric Reinforcement Learning Agentscited by 42
  16. 2023SayTap: Language to Quadrupedal Locomotioncited by 40
  17. 2022Large Language Models can Implement Policy Iterationcited by 31
  18. 2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agentscited by 30
  19. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  20. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approachcited by 28
  21. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  22. 2024Learning Reward for Robot Skills Using Large Language Models via Self-Alignmentcited by 19
  23. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  24. 2025A Survey on Large Language Models for Automated Planningcited by 17
  25. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Modelscited by 15
  26. 2023LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agentscited by 11
  27. 2024Language-guided Skill Learning with Temporal Variational Inferencecited by 11
  28. 2023AdaRefiner: Refining Decisions of Language Models with Adaptive Feedbackcited by 10
  29. 2024PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learningcited by 8
  30. 2018Leveraging human knowledge in tabular reinforcement learning: A study of human subjectscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 4.

10 papers.

  1. 2025AutoEDA: Enabling EDA Flow Automation through Microservice-Based LLM Agentscited by 6
  2. 2025HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Drivingcited by 4
  3. 2025ASTREA: Introducing Agentic Intelligence for Orbital Thermal Autonomycited by 2
  4. 2025Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement Learningcited by 1
  5. 2025Goal-Guided Efficient Exploration via Large Language Model in Reinforcement Learningcited by 0
  6. 2025LLM-Driven Composite Neural Architecture Search for Multi-Source RL State Encodingcited by 0
  7. 2026VLM-Guided Experience Replaycited by 0
  8. 2026MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidancecited by 0
  9. 2026When Adaptive Rewards Hurt: Causal Probing and the Switching-Stability Dilemma in LLM-Guided LEO Satellite Schedulingcited by 0
  10. 2026Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making
  2. 2026Hybrid LLM-Augmented Reinforcement Learning Agents for Complex Sequential Decision Tasks
  3. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
  4. 2025Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models
  5. 2024Code as Reward: Empowering Reinforcement Learning with VLMs
  6. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
  7. 2025Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation
  8. 2026Rethinking Agentic Reinforcement Learning In Large Language Models