mlsanityData to 7 Oct 2026

ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models

arXiv 2403.09583 ↗First posted 14 Mar 2024cs.ROcited by 19

Not on the map. Why?

Abstract (arXiv)

In robot manipulation, Reinforcement Learning (RL) often suffers from low sample efficiency and uncertain convergence, especially in large observation and action spaces. Foundation Models (FMs) offer an alternative, demonstrating promise in zero-shot and few-shot settings. However, they can be unreliable due to limited physical and spatial understanding. We introduce ExploRLLM, a method that combines the strengths of both paradigms. In our approach, FMs improve RL convergence by generating policy code and efficient representations, while a residual RL agent compensates for the FMs' limited physical understanding. We show that ExploRLLM outperforms both policies derived from FMs and RL baselines in table-top manipulation tasks. Additionally, real-world experiments show that the policies exhibit promising zero-shot sim-to-real transfer. Supplementary material is available at https://explorllm.github.io.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Mar 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 10; 2026: 5.

19 papers.

Sort:

By area:

  1. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  2. 2025Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Modelscited by 7
  3. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
  4. 2025Swarms of Large Language Model Agents for Protein Sequence Design with Experimental Validationcited by 5
  5. 2024Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videoscited by 3
  6. 2024MISR: Measuring Instrumental Self-Reasoning in Frontier Modelscited by 2
  7. 2025LLM-based Interactive Imitation Learning for Robotic Manipulationcited by 2
  8. 2025Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Modelscited by 1
  9. 2025APEX: Empowering LLMs with Physics-Based Task Planning for Real-time Insightcited by 1
  10. 2025Population-Coded Spiking Neural Networks for High-Dimensional Robotic Controlcited by 1
  11. 2024An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problemscited by 0
  12. 2025Towards a Reward-Free Reinforcement Learning Framework for Vehicle Controlcited by 0
7 more
  1. 2025Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusingcited by 0
  2. 2025Curriculum-Based Multi-Tier Semantic Exploration via Deep Reinforcement Learningcited by 0
  3. 2025Agent2: An Agent-Generates-Agent Framework for Reinforcement Learning Automationcited by 0
  4. 2026VLM-Guided Experience Replaycited by 0
  5. 2026Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Controlcited by 0
  6. 2026Reinforcement Learning from Cross-domain Videos with Video Prediction Modelcited by 0
  7. 2026Prompt-Driven Exploration: Language as an Exploration Space for VLA Reinforcement Learningcited by 0