mlsanityData to 7 Oct 2026

MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning

arXiv 2409.02714 ↗First posted 2 Sep 2024cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

In visual Reinforcement Learning (RL), learning from pixel-based observations poses significant challenges on sample efficiency, primarily due to the complexity of extracting informative state representations from high-dimensional data. Previous methods such as contrastive-based approaches have made strides in improving sample efficiency but fall short in modeling the nuanced evolution of states. To address this, we introduce MOOSS, a novel framework that leverages a temporal contrastive objective with the help of graph-based spatial-temporal masking to explicitly model state evolution in visual RL. Specifically, we propose a self-supervised dual-component strategy that integrates (1) a graph construction of pixel-based observations for spatial-temporal masking, coupled with (2) a multi-level contrastive learning mechanism that enriches state representations by emphasizing temporal continuity and change of states. MOOSS advances the understanding of state dynamics by disrupting and learning from spatial-temporal correlations, which facilitates policy learning. Our comprehensive evaluation on multiple continuous and discrete control benchmarks shows that MOOSS outperforms previous state-of-the-art visual RL methods in terms of sample efficiency, demonstrating the effectiveness of our method. Our code is released at https://github.com/jsun57/MOOSS.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 62 of its 78 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

62 of 78 references.

References · 62

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
38 more
  1. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2018DeepMind Control Suitecited by 935
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  7. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  10. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  12. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  13. 2019Model-Based Reinforcement Learning for Ataricited by 495
  14. 2020Reinforcement Learning with Augmented Datacited by 452
  15. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  16. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  17. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  18. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  19. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  20. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  21. 2017Noisy Networks for Explorationcited by 262
  22. 2020Self-supervised Co-training for Video Representation Learningcited by 232
  23. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  24. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  25. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  26. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  27. 2022Masked World Models for Visual Controlcited by 169
  28. 2019Unsupervised State Representation Learning in Ataricited by 144
  29. 2019When to use parametric models in reinforcement learning?cited by 118
  30. 2021APS: Active Pretraining with Successor Featurescited by 109
  31. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  32. 2021InfoGCL: Information-Aware Graph Contrastive Learningcited by 105
  33. 2022ConvMAE: Masked Convolution Meets Masked Autoencoderscited by 105
  34. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 101
  35. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  36. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  37. 2020Deep Reinforcement and InfoMax Learningcited by 79
  38. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Towards Efficient Large Scale Spatial-Temporal Time Series Forecasting via Improved Inverted Transformerscited by 1
  2. 2025Revealing Microscopic Objects in Fluorescence Live Imaging by Video-to-video Translation Based on A Spatial-temporal Generative Adversarial Networkcited by 0
  3. 2026Learning Dynamic Neural Evidence Representations for Time-Adaptive Brain-Computer Interfacescited by 0