mlsanityData to 14 Oct 2026

Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models

arXiv 2112.01163 ↗First posted 2 Dec 2021cs.LG cs.AI cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Modeling the world can benefit robot learning by providing a rich training signal for shaping an agent's latent state space. However, learning world models in unconstrained environments over high-dimensional observation spaces such as images is challenging. One source of difficulty is the presence of irrelevant but hard-to-model background distractions, and unimportant visual details of task-relevant entities. We address this issue by learning a recurrent latent dynamics model which contrastively predicts the next observation. This simple model leads to surprisingly robust robotic control even with simultaneous camera, background, and color distractions. We outperform alternatives such as bisimulation methods which impose state-similarity measures derived from divergence in future reward or future optimal actions. We obtain state-of-the-art results on the Distracting Control Suite, a challenging benchmark for pixel-based robotic control.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

42 of 56 references.

References · 42

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  9. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  10. 2017The Kinetics Human Action Video Datasetcited by 1,909
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  12. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
30 more
  1. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  4. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  5. 2018DeepMind Control Suitecited by 956
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  9. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  10. 2020Reinforcement Learning with Augmented Datacited by 468
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  12. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  13. 2015Skip-Thought Vectorscited by 322
  14. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  15. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  16. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  17. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  18. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  19. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  20. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  21. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  22. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
  23. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
  24. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  25. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 55
  26. 2020Neuroevolution of Self-Interpretable Agentscited by 49
  27. 2018Neural Predictive Belief Representationscited by 40
  28. 2019Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Controlcited by 18
  29. 2020Control-Aware Representations for Model-based Reinforcement Learningcited by 8
  30. 2021Unlocking Pixels for Reinforcement Learning via Implicit Attentioncited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Combining Reconstruction and Contrastive Methods for Multimodal Representations in RLcited by 9
  2. 2024KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertaintycited by 9

Similar, not cited

Close in wording, no citation either way

  1. 2025Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control
  2. 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
  3. 2025Pixel Motion Diffusion is What We Need for Robot Control
  4. 2026Mask World Model: Predicting What Matters for Robust Robot Policy Learning
  5. 2026Contrastive World Models
  6. 2026Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learning
  7. 2025RoDyn: Taming Interactive Robot-Dynamic 2.5D World Model for Robotic Manipulation
  8. 2020Planning from Pixels using Inverse Dynamics Models