mlsanityData to 7 Oct 2026

Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

arXiv 1812.00568 ↗First posted 3 Dec 2018cs.RO cs.AI cs.CVcited by 220

Not on the map. Why?

Cited by 80 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep reinforcement learning (RL) algorithms can learn complex robotic skills from raw sensory inputs, but have yet to achieve the kind of broad generalization and applicability demonstrated by deep learning methods in supervised domains. We present a deep RL method that is practical for real-world robotics tasks, such as robotic manipulation, and generalizes effectively to never-before-seen tasks and objects. In these settings, ground truth reward signals are typically unavailable, and we therefore propose a self-supervised model-based approach, where a predictive model learns to directly predict the future from raw sensory readings, such as camera images. At test time, we explore three distinct goal specification methods: designated pixels, where a user specifies desired object manipulation tasks by selecting particular pixels in an image and corresponding goal positions, goal images, where the desired goal state is specified with an image, and image classifiers, which define spaces of goal states. Our deep predictive models are trained using data collected autonomously and continuously by a robot interacting with hundreds of objects, without human supervision. We demonstrate that visual MPC can generalize to never-before-seen objects---both rigid and deformable---and solve a range of user-defined object manipulation tasks using the same model.

Key facts. Cited by 220 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 6; 2021: 41; 2022: 25; 2023: 36; 2024: 23; 2025: 33; 2026: 55.

220 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  3. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  4. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  5. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  6. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  7. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  8. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  9. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  10. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  11. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  12. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
38 more
  1. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  2. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  3. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  4. 2022Masked World Models for Visual Controlcited by 169
  5. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  6. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  7. 2023Structured World Models from Human Videoscited by 144
  8. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  9. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  10. 2021Discovering and Achieving Goals via World Modelscited by 131
  11. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  12. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  13. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  14. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  15. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  16. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  17. 2023On Bringing Robots Homecited by 101
  18. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  19. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  20. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  21. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  22. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 83
  23. 2022You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstrationcited by 79
  24. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  25. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  26. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  27. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  28. 2022A Survey on Model-based Reinforcement Learningcited by 65
  29. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RLcited by 65
  30. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  31. 2021Learning Visible Connectivity Dynamics for Cloth Smoothingcited by 61
  32. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  33. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 56
  34. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  35. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  36. 2021Offline Reinforcement Learning with Reverse Model-based Imaginationcited by 47
  37. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticscited by 47
  38. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 46

50 most cited of 220 All 220 (JSON)