mlsanityData to 7 Oct 2026

Target-absent Human Attention

arXiv 2207.01166 ↗First posted 4 Jul 2022cs.CV cs.AIcited by 20

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

The prediction of human gaze behavior is important for building human-computer interactive systems that can anticipate a user's attention. Computer vision models have been developed to predict the fixations made by people as they search for target objects. But what about when the image has no target? Equally important is to know how people search when they cannot find a target, and when they would stop searching. In this paper, we propose the first data-driven computational model that addresses the search-termination problem and predicts the scanpath of search fixations made by people searching for targets that do not appear in images. We model visual search as an imitation learning problem and represent the internal knowledge that the viewer acquires through fixations using a novel state representation that we call Foveated Feature Maps (FFMs). FFMs integrate a simulated foveated retina into a pretrained ConvNet that produces an in-network feature pyramid, all with minimal computational overhead. Our method integrates FFMs as the state representation in inverse reinforcement learning. Experimentally, we improve the state of the art in predicting human target-absent search behavior on the COCO-Search18 dataset

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 52 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

26 of 52 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2016Layer Normalizationcited by 5,579
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  8. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2016Generative Adversarial Imitation Learningcited by 1,576
  11. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  12. 2019Objects as Pointscited by 1,172
14 more
  1. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  2. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  4. 2019Panoptic Feature Pyramid Networkscited by 644
  5. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  6. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  7. 2016What do different evaluation metrics tell us about saliency models?cited by 184
  8. 2014Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNetcited by 58
  9. 2021DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modelingcited by 55
  10. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learningcited by 53
  11. 2021State-of-the-Art in Human Scanpath Predictioncited by 22
  12. 2014Object Detection Through Exploration With A Foveated Visual Fieldcited by 14
  13. 2020Predicting Goal-directed Attention Control Using Inverse-Reinforcement Learningcited by 8
  14. 2019Foveated image processing for faster object detection and recognition in embedded systems using deep convolutional neural networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 7; 2026: 2.

20 papers.

Sort:

By area:

  1. 2023Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attentioncited by 37
  2. 2023Unifying Top-down and Bottom-up Scanpath Prediction Using Transformerscited by 22
  3. 2024Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognitioncited by 17
  4. 2024GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpathscited by 15
  5. 2024Beyond Average: Individualized Visual Scanpath Predictioncited by 12
  6. 2023UniAR: A Unified model for predicting human Attention and Responses on visual contentcited by 7
  7. 2025Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Predictioncited by 7
  8. 2024Look Hear: Gaze Prediction for Speech-directed Human Attentioncited by 5
  9. 2025Few-shot Personalized Scanpath Predictioncited by 5
  10. 2025DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Imagescited by 5
  11. 2024TPP-Gaze: Modelling Gaze Dynamics in Space and Time with Neural Temporal Point Processescited by 4
  12. 2025Measuring and Predicting Where and When Pathologists Focus their Visual Attention while Grading Whole Slide Images of Cancercited by 3
8 more
  1. 2025Human Scanpath Prediction in Target-Present Visual Search with Semantic-Foveal Bayesian Attentioncited by 2
  2. 2024Driver Attention Tracking and Analysiscited by 1
  3. 2024GazeSearch: Radiology Findings Search Benchmarkcited by 1
  4. 2025Unified Attention Modeling for Efficient Free-Viewing and Visual Search via Shared Representationscited by 1
  5. 2024Unified Dynamic Scanpath Predictors Outperform Individually Trained Neural Modelscited by 0
  6. 2025EgoCampus: Egocentric Pedestrian Eye Gaze Model and Datasetcited by 0
  7. 2026EVA: Bridging Performance and Human Alignment in Hard-Attention Vision Models for Image Classificationcited by 0
  8. 2026Gaze-to-text Generation: Beyond Categorical Decoding of Human Attentioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans
  2. 2026Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learning
  3. 2026Emergent Goal-Directed Attention in Large Vision-Language Models
  4. 2021Imitation Learning with Human Eye Gaze via Multi-Objective Prediction
  5. 2026Why We Look Where We Look: Emergent Human-like Fixations of a Foveated Visual Language Model Maximizing Scene Understanding
  6. 2020Efficiently Guiding Imitation Learning Agents with Human Gaze
  7. 2020Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Games
  8. 2026Gaze-Regularized VLMs for Ego-Centric Behavior Understanding