mlsanityData to 7 Oct 2026

Critic Guided Segmentation of Rewarding Objects in First-Person Views

arXiv 2107.09540 ↗First posted 20 Jul 2021cs.CV cs.AIcited by 12

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

This work discusses a learning approach to mask rewarding objects in images using sparse reward signals from an imitation learning dataset. For that, we train an Hourglass network using only feedback from a critic model. The Hourglass network learns to produce a mask to decrease the critic's score of a high score image and increase the critic's score of a low score image by swapping the masked areas between these two images. We trained the model on an imitation learning dataset from the NeurIPS 2020 MineRL Competition Track, where our model learned to mask rewarding objects in a complex interactive 3D environment with a sparse reward signal. This approach was part of the 1st place winning solution in this competition. Video demonstration and code: https://rebrand.ly/critic-guided-segmentation

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 8 of its 23 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

8 of 23 references.

References · 8

  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  3. 2019Model-Based Reinforcement Learning for Ataricited by 495
  4. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  5. 2017Visualizing and Understanding Atari Agentscited by 141
  6. 2020Solving Physics Puzzles by Reasoning about Pathscited by 9
  7. 2019Modularization of End-to-End Learning: Case Study in Arcade Gamescited by 6
  8. 2021Towards robust and domain agnostic reinforcement learning competitionscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 3; 2024: 3; 2025: 1.

12 papers.

  1. 2023UniTeam: Open Vocabulary Mobile Manipulation Challengecited by 15
  2. 2022Behavioral Cloning via Search in Video PreTraining Latent Spacecited by 9
  3. 2024Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challengecited by 9
  4. 2022Planning with RL and episodic-memory behavioral priorscited by 8
  5. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  6. 2023Language-Conditioned Semantic Search-Based Policy for Robotic Manipulation Taskscited by 5
  7. 2021Transfer Learning with Jukebox for Music Source Separationcited by 4
  8. 2022YOLO -- You only look 10647 timescited by 4
  9. 2022Solving Learn-to-Race Autonomous Racing Challenge by Planning in Latent Spacecited by 4
  10. 2023Shape complexity estimation using VAEcited by 4
  11. 2024Lane Graph Extraction from Aerial Imagery via Lane Segmentation Refinement with Diffusion Modelscited by 4
  12. 2024STEVE-Audio: Expanding the Goal Conditioning Modalities of Embodied Agents in Minecraftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Visual Imitation Learning with Patch Rewards
  2. 2026Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models
  3. 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning
  4. 2021Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning
  5. 2026Interleaved Projected Gradient Descent for Safe Imitation Learning
  6. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  7. 2026Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
  8. 2024Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting