mlsanityData to 14 Oct 2026

Sequential Action-Induced Invariant Representation for Reinforcement Learning

arXiv 2309.12628 ↗First posted 22 Sep 2023cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

How to accurately learn task-relevant state representations from high-dimensional observations with visual distractions is a realistic and challenging problem in visual reinforcement learning. Recently, unsupervised representation learning methods based on bisimulation metrics, contrast, prediction, and reconstruction have shown the ability for task-relevant information extraction. However, due to the lack of appropriate mechanisms for the extraction of task information in the prediction, contrast, and reconstruction-related approaches and the limitations of bisimulation-related methods in domains with sparse rewards, it is still difficult for these methods to be effectively extended to environments with distractions. To alleviate these problems, in the paper, the action sequences, which contain task-intensive signals, are incorporated into representation learning. Specifically, we propose a Sequential Action--induced invariant Representation (SAR) method, in which the encoder is optimized by an auxiliary learner to only preserve the components that follow the control signals of sequential actions, so the agent can be induced to learn the robust representation against distractions. We conduct extensive experiments on the DeepMind Control suite tasks with distractions while achieving the best performance over strong baselines. We also demonstrate the effectiveness of our method at disregarding task-irrelevant information by deploying SAR to real-world CARLA-based autonomous driving with natural distractions. Finally, we provide the analysis results of generalization drawn from the generalization decay and t-SNE visualization. Code and demo videos are available at https://github.com/DMU-XMU/SAR.git.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 22 Sep 2023.

Builds on

39 of 50 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  8. 2020Reformer: The Efficient Transformercited by 1,668
  9. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  10. 2018DeepMind Control Suitecited by 956
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
27 more
  1. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  2. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  3. 2020Reinforcement Learning with Augmented Datacited by 468
  4. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  5. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  6. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  7. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  8. 2019Causality for Machine Learningcited by 243
  9. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  10. 2022Masked World Models for Visual Controlcited by 170
  11. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 149
  12. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  13. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  14. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
  15. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 68
  16. 2021Learning Task Informed Abstractionscited by 58
  17. 2021Towards Robust Bisimulation Metric Learningcited by 49
  18. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 45
  19. 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
  20. 2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
  21. 2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31
  22. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  23. 2021Cross-Trajectory Representation Learning for Zero-Shot Generalization in RLcited by 20
  24. 2022CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Drivingcited by 16
  25. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  26. 2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
  27. 2022Learning Generalizable Representations for Reinforcement Learning via Adaptive Meta-learner of Behavioral Similaritiescited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024Episodic Reinforcement Learning with Expanded State-reward Spacecited by 3
  2. 2025Learning Fused State Representations for Control from Multi-View Observationscited by 1
  3. 2024Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applicationscited by 0