# DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstruction

arXiv: https://arxiv.org/abs/2407.00633 · HTML: https://mlsanity.com/paper/2407.00633/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 30 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 18 of 18)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels](https://mlsanity.com/paper/2004.13649/) (2020)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Learning Invariant Representations for Reinforcement Learning without Reconstruction](https://mlsanity.com/paper/2006.10742/) (2020)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning](https://mlsanity.com/paper/2107.09645/) (2021)
- [Improving Sample Efficiency in Model-Free Reinforcement Learning from Images](https://mlsanity.com/paper/1910.01741/) (2019)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixels](https://mlsanity.com/paper/2101.02722/) (2021)
- [Transformers for One-Shot Visual Imitation](https://mlsanity.com/paper/2011.05970/) (2020)
- [Mask-based Latent Reconstruction for Reinforcement Learning](https://mlsanity.com/paper/2201.12096/) (2022)
- [Unsupervised Visual Attention and Invariance for Reinforcement Learning](https://mlsanity.com/paper/2104.02921/) (2021)
- [Conditional Mutual Information for Disentangled Representations in Reinforcement Learning](https://mlsanity.com/paper/2305.14133/) (2023)
- [Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2207.05480/) (2022)
- [Efficient RL via Disentangled Environment and Agent Representations](https://mlsanity.com/paper/2309.02435/) (2023)
- [On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learning](https://mlsanity.com/paper/2001.07973/) (2020)

## Built on by (most cited first, 6 of 6)

- [Zero-Shot Generalization of Vision-Based RL Without Data Augmentation](https://mlsanity.com/paper/2410.07441/) (2024)
- [Grounding Bodily Awareness in Visual Representations for Efficient Policy Learning](https://mlsanity.com/paper/2505.18487/) (2025)
- [Multimodal Fusion for Sim2real Transfer in Visual Reinforcement Learning](https://mlsanity.com/paper/2507.09180/) (2025)
- [CDE: Concept-Driven Exploration for Reinforcement Learning](https://mlsanity.com/paper/2510.08851/) (2025)
- [Few-Shot Adaptation to Non-Stationary Environments via Latent Trend Embedding for Robotics](https://mlsanity.com/paper/2603.10373/) (2026)
- [BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Models](https://mlsanity.com/paper/2606.16489/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) algorithms can learn robotic control tasks from visual observations, but they often require a large amount of data, especially when the visual scene is complex and unstructured. In this paper, we explore how the agent's knowledge of its shape can improve the sample efficiency of visual RL methods. We propose a novel method, Disentangled Environment and Agent Representations (DEAR), that uses the segmentation mask of the agent as supervision to learn disentangled representations of the environment and the agent through feature separation constraints. Unlike previous approaches, DEAR does not require reconstruction of visual observations. These representations are then used as an auxiliary loss to the RL objective, encouraging the agent to focus on the relevant features of the environment. We evaluate DEAR on two challenging benchmarks: Distracting DeepMind control suite and Franka Kitchen manipulation tasks. Our findings demonstrate that DEAR surpasses state-of-the-art methods in sample efficiency, achieving comparable or superior performance with reduced parameters. Our results indicate that integrating agent knowledge into visual RL methods has the potential to enhance their learning efficiency and robustness.