mlsanityData to 7 Oct 2026

Efficient RL via Disentangled Environment and Agent Representations

arXiv 2309.02435 ↗First posted 5 Sep 2023cs.LG cs.AI cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Agents that are aware of the separation between themselves and their environments can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, such as its shape or mask, which is often inexpensive to obtain. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, Structured Environment-Agent Representations, outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots. Website at https://sear-rl.github.io/

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

45 of 53 references.

References · 45

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2017Mask R-CNNcited by 7,259
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2015Continuous control with deep reinforcement learningcited by 3,279
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  12. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
33 more
  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  2. 2020Mastering Atari with Discrete World Modelscited by 990
  3. 2018DeepMind Control Suitecited by 935
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  7. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  9. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  10. 2020Reinforcement Learning with Augmented Datacited by 452
  11. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  12. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  13. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  14. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  15. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  16. 2022Masked Visual Pre-training for Motor Controlcited by 246
  17. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  18. 2017One-Shot Imitation Learningcited by 232
  19. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  20. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  21. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  22. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  23. 2022Human-to-Robot Imitation in the Wildcited by 157
  24. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  25. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 137
  26. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  27. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
  28. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  29. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  30. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  31. 2020Transformers for One-Shot Visual Imitationcited by 77
  32. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
  33. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 2; 2026: 1.

11 papers.

  1. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  2. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  3. 2024Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedbackcited by 20
  4. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  5. 2024EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generationcited by 8
  6. 2024DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstructioncited by 5
  7. 2024Learning Off-policy with Model-based Intrinsic Motivation For Active Online Explorationcited by 1
  8. 2024Ego-Foresight: Self-supervised Learning of Agent-Aware Representations for Improved RLcited by 0
  9. 2025Grounding Bodily Awareness in Visual Representations for Efficient Policy Learningcited by 0
  10. 2025Multimodal Fusion for Sim2real Transfer in Visual Reinforcement Learningcited by 0
  11. 2026BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Modelscited by 0