mlsanityData to 7 Oct 2026

Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods

arXiv 2404.00282 ↗First posted 30 Mar 2024cs.LG cs.AI cs.CLcited by 114

Not on the map. Why?

Cited by 27 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

With extensive pre-trained knowledge and high-level general capabilities, large language models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in aspects such as multi-task learning, sample efficiency, and high-level task planning. In this survey, we provide a comprehensive review of the existing literature in LLM-enhanced RL and summarize its characteristics compared to conventional RL methods, aiming to clarify the research scope and directions for future studies. Utilizing the classical agent-environment interaction paradigm, we propose a structured taxonomy to systematically categorize LLMs' functionalities in RL, including four roles: information processor, reward designer, decision-maker, and generator. For each role, we summarize the methodologies, analyze the specific RL challenges that are mitigated, and provide insights into future directions. Lastly, a comparative analysis of each role, potential applications, prospective opportunities, and challenges of the LLM-enhanced RL are discussed. By proposing this taxonomy, we aim to provide a framework for researchers to effectively leverage LLMs in the RL field, potentially accelerating RL applications in complex applications such as robotics, autonomous driving, and energy systems.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 142 of its 197 references that are in the map. First posted on arXiv 30 Mar 2024.

Builds on

142 of 197 references.

References · 142

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  11. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  12. 2020Scaling Laws for Neural Language Modelscited by 6,411
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  9. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  13. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  14. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  15. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  16. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  17. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  18. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  19. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  20. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  21. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  22. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  23. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  24. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  25. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  26. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  27. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  28. 2020Mastering Atari with Discrete World Modelscited by 990
  29. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  30. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  31. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  32. 2022Galactica: A Large Language Model for Sciencecited by 736
  33. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  34. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  35. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  36. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  37. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  38. 2023Bias and Fairness in Large Language Models: A Surveycited by 626

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 63; 2026: 29.

114 papers.

Sort:

By area:

  1. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  2. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  3. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 82
  4. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  5. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  6. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  7. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  8. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 34
  9. 2025Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Modelscited by 33
  10. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  11. 2024iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvementcited by 23
  12. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
38 more
  1. 2024Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learningcited by 22
  2. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 20
  3. 2025WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesiscited by 18
  4. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  5. 2024ElecBench: a Power Dispatch Evaluation Benchmark for Large Language Modelscited by 12
  6. 2025GTM: Simulating the World of Tools for AI Agentscited by 12
  7. 2025Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approachcited by 11
  8. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedbackcited by 10
  9. 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learningcited by 10
  10. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9
  11. 2025LERO: LLM-driven Evolutionary framework with Hybrid Rewards and Enhanced Observation for Multi-Agent Reinforcement Learningcited by 9
  12. 2025MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigmcited by 9
  13. 2026STAPO: Stabilizing Reinforcement Learning for LLMs by Silencing Rare Spurious Tokenscited by 9
  14. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  15. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8
  16. 2025ReDit: Reward Dithering for Improved LLM Policy Optimizationcited by 8
  17. 2025From Easy to Hard: The MIR Benchmark for Progressive Interleaved Multi-Image Reasoningcited by 8
  18. 2025MedBench v4: A Robust and Scalable Benchmark for Evaluating Chinese Medical Language Models, Multimodal Models, and Intelligent Agentscited by 8
  19. 2025Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learningcited by 7
  20. 2024Selective Exploration and Information Gathering in Search and Rescue Using Hierarchical Learning Guided by Natural Language Inputcited by 6
  21. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
  22. 2025DeepSeek-Inspired Exploration of RL-based LLMs and Synergy with Wireless Networks: A Surveycited by 6
  23. 2025A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimulicited by 6
  24. 2025DSADF: Thinking Fast and Slow for Decision Makingcited by 6
  25. 2025Efficient Reinforcement Learning with Semantic and Token Entropy for LLM Reasoningcited by 6
  26. 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agentscited by 6
  27. 2024Large Language Models are Biased Reinforcement Learnerscited by 5
  28. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 5
  29. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 5
  30. 2024Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategiescited by 4
  31. 2025LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Modelscited by 4
  32. 2025LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Modelscited by 4
  33. 2025HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Drivingcited by 4
  34. 2025Improving LLM Agents with Reinforcement Learning on Cryptographic CTF Challengescited by 4
  35. 2025LLM-Driven Intrinsic Motivation for Sparse Reward Reinforcement Learningcited by 4
  36. 2025UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancementcited by 4
  37. 2026AceGRPO: Adaptive Curriculum Enhanced Group Relative Policy Optimization for Autonomous Machine Learning Engineeringcited by 4
  38. 2024Guiding Reinforcement Learning Using Uncertainty-Aware Large Language Modelscited by 3

50 most cited of 114 All 114 (JSON)