mlsanityData to 7 Oct 2026

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

arXiv 1506.07365 ↗First posted 24 Jun 2015cs.LG cs.CV stat.MLcited by 266

Not on the map. Why?

Cited by 48 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Embed to Control (E2C), a method for model learning and control of non-linear dynamical systems from raw pixel images. E2C consists of a deep generative model, belonging to the family of variational autoencoders, that learns to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear. Our model is derived directly from an optimal control formulation in latent space, supports long-term prediction of image sequences and exhibits strong performance on a variety of complex control problems.

Key facts. Cited by 266 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 3; 2020: 3; 2021: 67; 2022: 51; 2023: 39; 2024: 39; 2025: 33; 2026: 31.

266 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  3. 2026Causal World Modeling for Robot Controlcited by 289
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  5. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  6. 2022Masked World Models for Visual Controlcited by 169
  7. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  8. 2021Discovering and Achieving Goals via World Modelscited by 131
  9. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  10. 2025Training Agents Inside of Scalable World Modelscited by 122
  11. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  12. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
38 more
  1. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  2. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  3. 2022Deep Hierarchical Planning from Pixelscited by 96
  4. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  5. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  6. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  7. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 78
  8. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  9. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  10. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  11. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70
  12. 2022A Survey on Model-based Reinforcement Learningcited by 65
  13. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  14. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  15. 2021DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Datacited by 62
  16. 2024NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Drivingcited by 61
  17. 2021Learning Task Informed Abstractionscited by 58
  18. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 54
  19. 2022Learning Symmetric Embeddings for Equivariant World Modelscited by 54
  20. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  21. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  22. 2021Towards Robust Bisimulation Metric Learningcited by 49
  23. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 49
  24. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
  25. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  26. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
  27. 2021Solving PDE-constrained Control Problems Using Operator Learningcited by 41
  28. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  29. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  30. 2025VRAG: Learning World Models for Interactive Video Generationcited by 37
  31. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 37
  32. 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
  33. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  34. 2023World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challengescited by 35
  35. 2023Deep networks for system identification: a Surveycited by 35
  36. 2026Hierarchical Planning with Latent World Modelscited by 34
  37. 2021Proper Value Equivalencecited by 33
  38. 2026Temporal Straightening for Latent Planningcited by 32

50 most cited of 266 All 266 (JSON)