mlsanityData to 7 Oct 2026

PcLast: Discovering Plannable Continuous Latent States

arXiv 2311.03534 ↗First posted 6 Nov 2023cs.LG cs.AI cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. While compact latent representations typically learned from variational autoencoders or inverse dynamics enable goal-conditioned decision making, they ignore state reachability, hampering their performance. In this paper, we learn a representation that associates reachable states together for effective planning and goal-conditioned policy learning. We first learn a latent representation with multi-step inverse dynamics (to remove distracting information), and then transform this representation to associate reachable states together in ℓ2 space. Our proposals are rigorously tested in various simulation testbeds. Numerical results in reward-based settings show significant improvements in sampling efficiency. Further, in reward-free settings this approach yields layered state abstractions that enable computationally efficient hierarchical planning for reaching ad hoc goals with zero additional samples.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

41 of 63 references.

References · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2016OpenAI Gymcited by 1,684
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2018Group Normalizationcited by 1,514
  9. 2018World Modelscited by 1,444
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  11. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  12. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
29 more
  1. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  2. 2017Hindsight Experience Replaycited by 1,096
  3. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  4. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  5. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  6. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  7. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  8. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  9. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  10. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  11. 2019Planning with Goal-Conditioned Policiescited by 144
  12. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  13. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
  14. 2019Exploring Model-based Planning with Policy Networkscited by 90
  15. 2020Sample-efficient Cross-Entropy Method for Real-time Planningcited by 90
  16. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  17. 2020Deep Reinforcement and InfoMax Learningcited by 79
  18. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  19. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  20. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 56
  21. 2019A Geometric Perspective on Optimal Representations for Reinforcement Learningcited by 54
  22. 2020Model-Based Visual Planning with Self-Supervised Functional Distancescited by 41
  23. 2020Planning from Pixels using Inverse Dynamics Modelscited by 36
  24. 2020Model-Predictive Control via Cross-Entropy and Gradient-Based Optimizationcited by 32
  25. 2022On the Generalization of Representations in Reinforcement Learningcited by 28
  26. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  27. 2023Representation Learning with Multi-Step Inverse Kinematics: An Efficient and Optimal Approach to Rich-Observation RLcited by 22
  28. 2023Bootstrapped Representations in Reinforcement Learningcited by 11
  29. 2023CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapsecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 3.

4 papers.

  1. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  2. 2026Predictive but Not Plannable: RC-aux for Latent World Modelscited by 16
  3. 2026Laplacian Representations for Decision-Time Planningcited by 4
  4. 2026When Does Predictive Inverse Dynamics Outperform Behavior Cloning?cited by 3