# ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning

arXiv: https://arxiv.org/abs/2011.06777 · HTML: https://mlsanity.com/paper/2011.06777/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [Self-supervised Reinforcement Learning with Independently Controllable Subgoals](https://mlsanity.com/paper/2109.04150/) (2021)
- [Visual Affordance Prediction for Guiding Robot Exploration](https://mlsanity.com/paper/2305.17783/) (2023)
- [Learning Domain Invariant Representations in Goal-conditioned Block MDPs](https://mlsanity.com/paper/2110.14248/) (2021)
- [Multi-Agent Variational Occlusion Inference Using People as Sensors](https://mlsanity.com/paper/2109.02173/) (2021)
- [Occlusion-Aware Crowd Navigation Using People as Sensors](https://mlsanity.com/paper/2210.00552/) (2022)
- [Learning Visual-Audio Representations for Voice-Controlled Robots](https://mlsanity.com/paper/2109.02823/) (2021)
- [Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning](https://mlsanity.com/paper/2307.10846/) (2023)
- [Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning](https://mlsanity.com/paper/2202.13624/) (2022)
- [A Brief Survey on Leveraging Large Scale Vision Models for Enhanced Robot Grasping](https://mlsanity.com/paper/2406.11786/) (2024)
- [Dynamic Object Masks as Goal Representations for Visual Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2510.06277/) (2025)

## Abstract (arXiv)

Current image-based reinforcement learning (RL) algorithms typically operate on the whole image without performing object-level reasoning. This leads to inefficient goal sampling and ineffective reward functions. In this paper, we improve upon previous visual self-supervised RL by incorporating object-level reasoning and occlusion reasoning. Specifically, we use unknown object segmentation to ignore distractors in the scene for better reward computation and goal generation; we further enable occlusion reasoning by employing a novel auxiliary loss and training scheme. We demonstrate that our proposed algorithm, ROLL (Reinforcement learning with Object Level Learning), learns dramatically faster and achieves better final performance compared with previous methods in several simulated visual control tasks. Project video and code are available at https://sites.google.com/andrew.cmu.edu/roll.