mlsanityData to 14 Oct 2026

CDE: Concept-Driven Exploration for Reinforcement Learning

arXiv 2510.08851 ↗First posted 9 Oct 2025cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Intelligent exploration remains a critical challenge in reinforcement learning (RL), especially in visual control tasks. Unlike low-dimensional state-based RL, visual RL must extract task-relevant structure from raw pixels, making exploration inefficient. We propose Concept-Driven Exploration (CDE), which leverages a pre-trained vision-language model (VLM) to generate object-centric visual concepts from textual task descriptions as weak, potentially noisy supervisory signals. Rather than directly conditioning on these noisy signals, CDE trains a policy to reconstruct the concepts via an auxiliary objective, learning general representations of the concepts and using reconstruction accuracy as an intrinsic reward to guide exploration toward task-relevant objects. Across five challenging simulated visual manipulation tasks, CDE achieves efficient, targeted exploration and remains robust to both synthetic errors and noisy VLM predictions. Finally, we demonstrate real-world transfer by deploying CDE on a Franka arm, attaining an 80% success rate in a real-world manipulation task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 46 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

32 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstruction30 Jun

Other references · 31

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2018Exploration by Random Network Distillationcited by 993
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  11. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  12. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
19 more
  1. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  2. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  3. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  4. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  5. 2016Learning to Navigate in Complex Environmentscited by 198
  6. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  7. 2017Deep Learning is Robust to Massive Label Noisecited by 165
  8. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 165
  9. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 156
  10. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 156
  11. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 134
  12. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 129
  13. 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priorscited by 51
  14. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 43
  15. 2021Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Reviewcited by 36
  16. 2024Exploration and Anti-Exploration with Distributional Random Network Distillationcited by 33
  17. 2023Concept Learning for Interpretable Multi-Agent Reinforcement Learningcited by 19
  18. 2023Efficient RL via Disentangled Environment and Agent Representationscited by 12
  19. 2024LICORICE: Label-Efficient Concept-Based Interpretable Reinforcement Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models
  2. 2025Discover, Learn, and Reinforce: Scaling Vision-Language-Action Pretraining with Diverse RL-Generated Trajectories
  3. 2026ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuning
  4. 2026DreamPlan: Efficient Reinforcement Fine-Tuning of Vision-Language Planners via Video World Models
  5. 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data
  6. 2026VLGOR: Visual-Language Knowledge Guided Offline Reinforcement Learning for Generalizable Agents
  7. 2025Curriculum-Based Multi-Tier Semantic Exploration via Deep Reinforcement Learning
  8. 2026RMTL: Reinforced Micro-task Learning for Long-Horizon Manipulation with VLM Rewards ↗