mlsanityData to 7 Oct 2026

Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models

arXiv 2311.02379 ↗First posted 4 Nov 2023cs.RO cs.AI cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) plays an important role in the robotic manipulation domain since it allows self-learning from trial-and-error interactions with the environment. Still, sample efficiency and reward specification seriously limit its potential. One possible solution involves learning from expert guidance. However, obtaining a human expert is impractical due to the high cost of supervising an RL agent, and developing an automatic supervisor is a challenging endeavor. Large Language Models (LLMs) demonstrate remarkable abilities to provide human-like feedback on user inputs in natural language. Nevertheless, they are not designed to directly control low-level robotic motions, as their pretraining is based on vast internet data rather than specific robotics data. In this paper, we introduce the Lafite-RL (Language agent feedback interactive Reinforcement Learning) framework, which enables RL agents to learn robotic tasks efficiently by taking advantage of LLMs' timely feedback. Our experiments conducted on RLBench tasks illustrate that, with simple prompt design in natural language, the Lafite-RL agent exhibits improved learning capabilities when guided by an LLM. It outperforms the baseline in terms of both learning efficiency and success rate, underscoring the efficacy of the rewards provided by an LLM.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 4 Nov 2023.

Builds on

33 of 45 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  10. 2023Mastering Diverse Domains through World Modelscited by 1,193
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
21 more
  1. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  2. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  3. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  4. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  5. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  6. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  7. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  8. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  9. 2023Reward Design with Language Modelscited by 257
  10. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  11. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  12. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 122
  13. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  14. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  15. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedbackcited by 56
  16. 2023Chat with the Environment: Interactive Multimodal Perception Using Large Language Modelscited by 42
  17. 2018DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedbackcited by 39
  18. 2021Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedbackcited by 34
  19. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  20. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  21. 2019Improving interactive reinforcement learning: What makes a good teacher?cited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 7; 2026: 5.

22 papers.

Sort:

By area:

  1. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  2. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  3. 2025Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learningcited by 21
  4. 2024Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMscited by 11
  5. 2024Large Language Models for Orchestrating Bimanual Robotscited by 10
  6. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedbackcited by 10
  7. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  8. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 5
  9. 2025LuciBot: Automated Robot Policy Learning from Generated Videoscited by 4
  10. 2024From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledgecited by 3
  11. 2024Guiding Reinforcement Learning Using Uncertainty-Aware Large Language Modelscited by 3
  12. 2025EmboMatrix: A Scalable Training-Ground for Embodied Decision-Makingcited by 3
10 more
  1. 2024Enhancing Q-Learning with Large Language Model Heuristicscited by 2
  2. 2025AED: Automatic Discovery of Effective and Diverse Vulnerabilities for Autonomous Driving Policy with Large Language Modelscited by 2
  3. 2025LLM-based Interactive Imitation Learning for Robotic Manipulationcited by 2
  4. 2025HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learningcited by 2
  5. 2026LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learningcited by 2
  6. 2026Automating Potential-based Reward Shaping with Vision Language Model Guidancecited by 1
  7. 2024Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Modelscited by 0
  8. 2026MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidancecited by 0
  9. 2026Hybrid Framework for Robotic Manipulation: Integrating Reinforcement Learning and Large Language Modelscited by 0
  10. 2026VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learningcited by 0