mlsanityData to 7 Oct 2026

LLM-Empowered State Representation for Reinforcement Learning

arXiv 2407.13237 ↗First posted 18 Jul 2024cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Conventional state representations in reinforcement learning often omit critical task-related details, presenting a significant challenge for value networks in establishing accurate mappings from states to task rewards. Traditional methods typically depend on extensive sample learning to enrich state representations with task-specific information, which leads to low sample efficiency and high time costs. Recently, surging knowledgeable large language models (LLM) have provided promising substitutes for prior injection with minimal human intervention. Motivated by this, we propose LLM-Empowered State Representation (LESR), a novel approach that utilizes LLM to autonomously generate task-related state representation codes which help to enhance the continuity of network mappings and facilitate efficient training. Experimental results demonstrate LESR exhibits high sample efficiency and outperforms state-of-the-art baselines by an average of 29% in accumulated reward in Mujoco tasks and 30% in success rates in Gym-Robotics tasks.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 73 of its 124 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

73 of 124 references.

References · 73

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2015Continuous control with deep reinforcement learningcited by 3,279
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2015Trust Region Policy Optimizationcited by 2,503
38 more
  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2016OpenAI Gymcited by 1,684
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  9. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  10. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  11. 2018Exploration by Random Network Distillationcited by 966
  12. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  13. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  14. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  15. 2019The StarCraft Multi-Agent Challengecited by 589
  16. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  17. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  18. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  19. 2016Learning Multiagent Communication with Backpropagationcited by 355
  20. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  21. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  22. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  23. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  24. 2019Challenges of Real-World Reinforcement Learningcited by 282
  25. 2023Reward Design with Language Modelscited by 257
  26. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  27. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  28. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  29. 2018Sorting out Lipschitz function approximationcited by 211
  30. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  31. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  32. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  33. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
  34. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  35. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  36. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  37. 2018State Representation Learning for Control: An Overviewcited by 125
  38. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 17; 2026: 10.

28 papers.

Sort:

By area:

  1. 2025RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Codecited by 28
  2. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  3. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  4. 2026Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Modelscited by 11
  5. 2025Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learningcited by 10
  6. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9
  7. 2025LEED: A Highly Efficient and Scalable LLM-Empowered Expert Demonstrations Framework for Multi-Agent Reinforcement Learningcited by 6
  8. 2025Text-to-Decision Agent: Offline Meta-Reinforcement Learning from Natural Language Supervisioncited by 5
  9. 2025Scalable UAV Multi-Hop Networking via Multi-Agent Reinforcement Learning with Large Language Modelscited by 4
  10. 2025LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Modelscited by 4
  11. 2025ORAN-GUIDE: RAG-Driven Prompt Learning for LLM-Augmented Reinforcement Learning in O-RAN Network Slicingcited by 3
  12. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
16 more
  1. 2025Prompt-Tuned LLM-Augmented DRL for Dynamic O-RAN Network Slicingcited by 2
  2. 2026LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learningcited by 2
  3. 2025Agentic Episodic Controlcited by 1
  4. 2026Large Language Model (LLM)-enabled Reinforcement Learning for Wireless Network Optimizationcited by 1
  5. 2026PriorZero: Bridging Language Priors and World Models for Decision Makingcited by 1
  6. 2026GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learningcited by 1
  7. 2025Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme Onecited by 0
  8. 2025MARCO: Meta-Reflection with Cross-Referencing for Code Reasoningcited by 0
  9. 2025LLM-Driven Policy Diffusion: Enhancing Generalization in Offline Reinforcement Learningcited by 0
  10. 2025Agent2: An Agent-Generates-Agent Framework for Reinforcement Learning Automationcited by 0
  11. 2025LLM-Driven Composite Neural Architecture Search for Multi-Source RL State Encodingcited by 0
  12. 2026Formulating Reinforcement Learning for Human-Robot Collaboration through Off-Policy Evaluationcited by 0
  13. 2026Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robotscited by 0
  14. 2026Discovering Reinforcement Learning Interfaces with Large Language Modelscited by 0
  15. 2026Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoningcited by 0
  16. 2026Select-to-Act: Hierarchical Reinforcement Learning via Adaptive Language Guidancecited by 0