mlsanityData to 7 Oct 2026

Temporal Predictive Coding For Model-Based Planning In Latent Space

arXiv 2106.07156 ↗First posted 14 Jun 2021cs.LG cs.AIcited by 54

Not on the map. Why?

Cited by 21 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

High-dimensional observations are a major challenge in the application of model-based reinforcement learning (MBRL) to real-world environments. To handle high-dimensional sensory inputs, existing approaches use representation learning to map high-dimensional observations into a lower-dimensional latent space that is more amenable to dynamics estimation and planning. In this work, we present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time. Since this approach focuses on encoding temporally-predictable information, we implicitly prioritize the encoding of task-relevant components over nuisance information within the environment that are provably task-irrelevant. By learning this representation in conjunction with a recurrent state space model, we can then perform planning in latent space. We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task. Our experiments show that our model is superior to existing methods in the challenging complex-background setting while remaining competitive with current state-of-the-art models in the standard setting.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

30 of 35 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2017The Kinetics Human Action Video Datasetcited by 1,894
  9. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  10. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  11. 2019Contrastive Multiview Codingcited by 1,297
  12. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
18 more
  1. 2018DeepMind Control Suitecited by 935
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  4. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  5. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  6. 2019On Variational Bounds of Mutual Informationcited by 550
  7. 2019Model-Based Reinforcement Learning for Ataricited by 495
  8. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  9. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  10. 2020A Theory of Usable Information Under Computational Constraintscited by 153
  11. 2018Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiencycited by 103
  12. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  13. 2017Robust Locally-Linear Controllable Embeddingcited by 37
  14. 2019Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Controlcited by 17
  15. 2020Predictive Coding for Locally-Linear Controlcited by 17
  16. 2020Contrastive Variational Reinforcement Learning for Complex Observationscited by 15
  17. 2020Control-Aware Representations for Model-based Reinforcement Learningcited by 8
  18. 2020Mutual Information Maximization for Robust Plannable Representationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 13; 2023: 12; 2024: 11; 2025: 8; 2026: 7.

54 papers.

Sort:

By area:

  1. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  2. 2022Masked World Models for Visual Controlcited by 169
  3. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  4. 2021DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representationscited by 73
  5. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  6. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
  7. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  8. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
  9. 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
  10. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  11. 2022Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Datacited by 27
  12. 2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
38 more
  1. 2023Learning World Models with Identifiable Factorizationcited by 23
  2. 2023HarmonyDream: Task Harmonization Inside World Modelscited by 22
  3. 2023GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Modelscited by 20
  4. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17
  5. 2026Predictive but Not Plannable: RC-aux for Latent World Modelscited by 16
  6. 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 15
  7. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
  8. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
  9. 2024MuDreamer: Learning Predictive World Models without Reconstructioncited by 13
  10. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 11
  11. 2023Goal-Conditioned Predictive Coding for Offline Reinforcement Learningcited by 10
  12. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
  13. 2022S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learningcited by 9
  14. 2023Combining Reconstruction and Contrastive Methods for Multimodal Representations in RLcited by 9
  15. 2024KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertaintycited by 9
  16. 2022Learning Robust Dynamics through Variational Sparse Gatingcited by 8
  17. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 7
  18. 2024Model-based Reinforcement Learning for Parameterized Action Spacescited by 7
  19. 2025Knowledge Retention for Continual Model-Based Reinforcement Learningcited by 6
  20. 2025Bootstrap Off-policy with World Modelcited by 6
  21. 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learningcited by 5
  22. 2023Task Aware Dreamer for Task Generalization in Reinforcement Learningcited by 5
  23. 2024Focus On What Matters: Separated Models For Visual-Based RL Generalizationcited by 5
  24. 2024Learning Abstract World Model for Value-preserving Planning with Optionscited by 4
  25. 2024Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractionscited by 4
  26. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RLcited by 4
  27. 2023Sample-efficient Real-time Planning with Curiosity Cross-Entropy Method and Contrastive Learningcited by 3
  28. 2024CoVO-MPC: Theoretical Analysis of Sampling-based MPC and Optimal Covariance Designcited by 3
  29. 2024State Chrono Representation for Enhancing Generalization in Reinforcement Learningcited by 3
  30. 2025Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractionscited by 3
  31. 2026Latent State Design for World Models under Sufficiency Constraintscited by 3
  32. 2021Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Modelscited by 2
  33. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 2
  34. 2026Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Predictioncited by 2
  35. 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistencycited by 2
  36. 2024LaPlaSS: Latent Space Planning for Stochastic Systemscited by 1
  37. 2025Predictive Learning in Energy-based Models with Attractor Structurescited by 1
  38. 2022Efficient Deep Reinforcement Learning with Predictive Processing Proximal Policy Optimizationcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Active Predictive Coding: A Unified Neural Framework for Learning Hierarchical World Models for Perception and Planning
  2. 2026Beyond a single latent space: a dual-latent world model for long-horizon planning
  3. 2026Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
  4. 2026Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
  5. 2020Planning from Pixels using Inverse Dynamics Models
  6. 2024Learning Latent Dynamic Robust Representations for World Models
  7. 2026In-Context Planning with Latent Temporal Abstractions
  8. 2026Learning Invariant Visual Representations for Planning with Joint-Embedding Predictive World Models