# Target-absent Human Attention

arXiv: https://arxiv.org/abs/2207.01166 · HTML: https://mlsanity.com/paper/2207.01166/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [IQ-Learn: Inverse soft-Q Learning for Imitation](https://mlsanity.com/paper/2106.12142/) (2021)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet](https://mlsanity.com/paper/1411.1045/) (2014)
- [DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling](https://mlsanity.com/paper/2105.12441/) (2021)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [State-of-the-Art in Human Scanpath Prediction](https://mlsanity.com/paper/2102.12239/) (2021)
- [Object Detection Through Exploration With A Foveated Visual Field](https://mlsanity.com/paper/1408.0814/) (2014)
- [Predicting Goal-directed Attention Control Using Inverse-Reinforcement Learning](https://mlsanity.com/paper/2001.11921/) (2020)
- [Foveated image processing for faster object detection and recognition in embedded systems using deep convolutional neural networks](https://mlsanity.com/paper/1908.09000/) (2019)

## Built on by (most cited first, 20 of 20)

- [Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention](https://mlsanity.com/paper/2303.15274/) (2023)
- [Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers](https://mlsanity.com/paper/2303.09383/) (2023)
- [Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognition](https://mlsanity.com/paper/2405.09931/) (2024)
- [GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths](https://mlsanity.com/paper/2408.02788/) (2024)
- [Beyond Average: Individualized Visual Scanpath Prediction](https://mlsanity.com/paper/2404.12235/) (2024)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction](https://mlsanity.com/paper/2507.23021/) (2025)
- [Look Hear: Gaze Prediction for Speech-directed Human Attention](https://mlsanity.com/paper/2407.19605/) (2024)
- [Few-shot Personalized Scanpath Prediction](https://mlsanity.com/paper/2504.05499/) (2025)
- [DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images](https://mlsanity.com/paper/2509.16767/) (2025)
- [TPP-Gaze: Modelling Gaze Dynamics in Space and Time with Neural Temporal Point Processes](https://mlsanity.com/paper/2410.23409/) (2024)
- [Measuring and Predicting Where and When Pathologists Focus their Visual Attention while Grading Whole Slide Images of Cancer](https://mlsanity.com/paper/2508.01668/) (2025)
- [Human Scanpath Prediction in Target-Present Visual Search with Semantic-Foveal Bayesian Attention](https://mlsanity.com/paper/2507.18503/) (2025)
- [Driver Attention Tracking and Analysis](https://mlsanity.com/paper/2404.07122/) (2024)
- [Unified Dynamic Scanpath Predictors Outperform Individually Trained Neural Models](https://mlsanity.com/paper/2405.02929/) (2024)
- [GazeSearch: Radiology Findings Search Benchmark](https://mlsanity.com/paper/2411.05780/) (2024)
- [Unified Attention Modeling for Efficient Free-Viewing and Visual Search via Shared Representations](https://mlsanity.com/paper/2506.02764/) (2025)
- [EgoCampus: Egocentric Pedestrian Eye Gaze Model and Dataset](https://mlsanity.com/paper/2512.07668/) (2025)
- [EVA: Bridging Performance and Human Alignment in Hard-Attention Vision Models for Image Classification](https://mlsanity.com/paper/2603.27340/) (2026)
- [Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention](https://mlsanity.com/paper/2607.23917/) (2026)

## Abstract (arXiv)

The prediction of human gaze behavior is important for building human-computer interactive systems that can anticipate a user's attention. Computer vision models have been developed to predict the fixations made by people as they search for target objects. But what about when the image has no target? Equally important is to know how people search when they cannot find a target, and when they would stop searching. In this paper, we propose the first data-driven computational model that addresses the search-termination problem and predicts the scanpath of search fixations made by people searching for targets that do not appear in images. We model visual search as an imitation learning problem and represent the internal knowledge that the viewer acquires through fixations using a novel state representation that we call Foveated Feature Maps (FFMs). FFMs integrate a simulated foveated retina into a pretrained ConvNet that produces an in-network feature pyramid, all with minimal computational overhead. Our method integrates FFMs as the state representation in inverse reinforcement learning. Experimentally, we improve the state of the art in predicting human target-absent search behavior on the COCO-Search18 dataset