mlsanityData to 7 Oct 2026

Planning from Pixels in Environments with Combinatorially Hard Search Spaces

arXiv 2110.06149 ↗First posted 12 Oct 2021cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

The ability to form complex plans based on raw visual input is a litmus test for current capabilities of artificial intelligence, as it requires a seamless combination of visual processing and abstract algorithmic execution, two traditionally separate areas of computer science. A recent surge of interest in this field brought advances that yield good performance in tasks ranging from arcade games to continuous control; these methods however do not come without significant issues, such as limited generalization capabilities and difficulties when dealing with combinatorially hard planning instances. Our contribution is two-fold: (i) we present a method that learns to represent its environment as a latent graph and leverages state reidentification to reduce the complexity of finding a good policy from exponential to linear (ii) we introduce a set of lightweight environments with an underlying discrete combinatorial structure in which planning is challenging even for humans. Moreover, we show that our methods achieves strong empirical generalization to variations in the environment, even across highly disadvantaged regimes, such as "one-shot" planning, or in an offline RL paradigm which only provides low-quality trajectories.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 12 Oct 2021.

Builds on

35 of 47 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2020Mastering Atari with Discrete World Modelscited by 990
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  11. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  12. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
23 more
  1. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  2. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  3. 2014Deep metric learning using Triplet networkcited by 421
  4. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  5. 2019Differentiation of Blackbox Combinatorial Solverscited by 224
  6. 2019Contrastive Learning of Structured World Modelscited by 206
  7. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  8. 2018Semi-parametric Topological Memory for Navigationcited by 203
  9. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  10. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  11. 2019Planning with Goal-Conditioned Policiescited by 144
  12. 2017Recurrent Environment Simulatorscited by 68
  13. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 67
  14. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  15. 2017Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundarycited by 58
  16. 2013A Survey on Metric Learning for Feature Vectors and Structured Datacited by 57
  17. 2020Path Planning using Neural A* Searchcited by 51
  18. 2018Robot Motion Planning in Learned Latent Spacescited by 40
  19. 2020Planning from Pixels using Inverse Dynamics Modelscited by 36
  20. 2020Understanding the Power and Limitations of Teaching with Imperfect Knowledgecited by 23
  21. 2019LEGO: Leveraging Experience in Roadmap Generation for Sampling-Based Planningcited by 19
  22. 2021Neuro-algorithmic Policies enable Fast Combinatorial Generalizationcited by 11
  23. 2018Deep sequential models for sampling-based planningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1; 2026: 2.

4 papers.

  1. 2025Contrastive Representations for Temporal Reasoningcited by 5
  2. 2026Graph World Models: Concepts, Taxonomy, and Future Directionscited by 5
  3. 2022Challenges to Solving Combinatorially Hard Long-Horizon Deep RL Taskscited by 4
  4. 2026Discrete World Models via Regularizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Sparse Planning in Visual World Models via Cost Gradients
  2. 2020Sparse Graphical Memory for Robust Planning
  3. 2025Visual Planning: Let's Think Only with Images
  4. 2022Deep Hierarchical Planning from Pixels
  5. 2020Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
  6. 2025Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search
  7. 2026Coding Agents for Generalized Task and Motion Planning Problems
  8. 2026Efficient Test-time Inference for Generative Planning Models with OCL Search