mlsanityData to 7 Oct 2026

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

arXiv 1806.10293 ↗First posted 27 Jun 2018cs.LG cs.AI cs.CVcited by 730

Not on the map. Why?

Cited by 137 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execute the desired grasp, our method enables closed-loop vision-based control, whereby the robot continuously updates its grasp strategy based on the most recent observations to optimize long-horizon grasp success. To that end, we introduce QT-Opt, a scalable self-supervised vision-based reinforcement learning framework that can leverage over 580k real-world grasp attempts to train a deep neural network Q-function with over 1.2M parameters to perform closed-loop, real-world grasping that generalizes to 96% grasp success on unseen objects. Aside from attaining a very high success rate, our method exhibits behaviors that are quite distinct from more standard grasping systems: using only RGB vision-based perception from an over-the-shoulder camera, our method automatically learns regrasping strategies, probes objects to find the most effective grasps, learns to reposition objects and perform other non-prehensile pre-grasp manipulations, and responds dynamically to disturbances and perturbations.

Key facts. Cited by 730 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 3; 2020: 8; 2021: 130; 2022: 115; 2023: 138; 2024: 134; 2025: 122; 2026: 80.

730 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  7. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2024Genie: Generative Interactive Environmentscited by 706
  11. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  12. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
38 more
  1. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  2. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  3. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  4. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  5. 2021Implicit Behavioral Cloningcited by 442
  6. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  7. 2023Augmented Language Models: a Surveycited by 398
  8. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  9. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  10. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  11. 2022Kubric: A scalable dataset generatorcited by 373
  12. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  13. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  14. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  15. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  16. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  17. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  18. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  19. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  20. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  21. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  22. 2022Masked Visual Pre-training for Motor Controlcited by 246
  23. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  24. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  25. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  26. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  27. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 213
  28. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  29. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  30. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  31. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  32. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  33. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  34. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  35. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  36. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  37. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
  38. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157

50 most cited of 730 All 730 (JSON)