mlsanityData to 7 Oct 2026

Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning

arXiv 2510.08442 ↗First posted 9 Oct 2025cs.CV cs.AI cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual Reinforcement Learning (RL) agents must learn to act based on high-dimensional image data where only a small fraction of the pixels is task-relevant. This forces agents to waste exploration and computational resources on irrelevant features, leading to sample-inefficient and unstable learning. To address this, inspired by human visual foveation, we introduce Gaze on the Prize. This framework augments visual RL with a learnable foveal attention mechanism (Gaze), guided by a self-supervised signal derived from the agent's experience pursuing higher returns (the Prize). Our key insight is that return differences reveal what matters most: If two similar representations produce different outcomes, their distinguishing features are likely task-relevant, and the gaze should focus on them accordingly. This is realized through return-guided contrastive learning that trains the attention to distinguish between the features relevant to success and failure. We group similar visual representations into positives and negatives based on their return differences and use the resulting labels to construct contrastive triplets. These triplets provide the training signal that teaches the attention mechanism to produce distinguishable representations for states associated with different outcomes. Our method achieves up to 2.52x improvement in sample efficiency and can solve challenging tasks from the ManiSkill3 benchmark that the baseline fails to learn, without modifying the underlying algorithm or hyperparameters.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

34 of 48 references.

References · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2018DeepMind Control Suitecited by 935
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  9. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  11. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  12. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
22 more
  1. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  2. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  3. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  4. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
  5. 2016DeepGaze II: Reading fixations from deep features trained on object recognitioncited by 54
  6. 2018Remember and Forget for Experience Replaycited by 42
  7. 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 37
  8. 2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
  9. 2025Vision in Action: Learning Active Perception from Human Demonstrationscited by 33
  10. 2020Masked Contrastive Representation Learning for Reinforcement Learningcited by 32
  11. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
  12. 2018PiCANet: Pixel-wise Contextual Attention Learning for Accurate Saliency Detectioncited by 28
  13. 2025Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loopcited by 23
  14. 2016Seeing with Humans: Gaze-Assisted Neural Image Captioningcited by 20
  15. 2020Tidying Deep Saliency Prediction Architecturescited by 20
  16. 2024Active Vision Might Be All You Need: Exploring Active Vision in Bimanual Robotic Manipulationcited by 20
  17. 2024ViSaRL: Visual Reinforcement Learning Guided by Human Saliencycited by 12
  18. 2025Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformerscited by 8
  19. 2023MENTOR: Human Perception-Guided Pretraining for Increased Generalizationcited by 4
  20. 2024Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractionscited by 4
  21. 2020Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learningcited by 1
  22. 2024Learning Gaussian Representation for Eye Fixation Predictioncited by 1

Built on by

Not cited yet.