# Forecasting User Attention During Everyday Mobile Interactions Using Device-Integrated and Wearable Sensors

arXiv: https://arxiv.org/abs/1801.06011 · HTML: https://mlsanity.com/paper/1801.06011/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2018.

## Built on by (most cited first, 6 of 6)

- [An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices](https://mlsanity.com/paper/2307.00122/) (2023)
- [Pose2Gaze: Eye-body Coordination during Daily Activities for Gaze Prediction from Full-body Poses](https://mlsanity.com/paper/2312.12042/) (2023)
- [Quantifying the Impact of Motion on 2D Gaze Estimation in Real-World Mobile Interactions](https://mlsanity.com/paper/2502.10570/) (2025)
- [Gaze Beyond the Frame: Forecasting Egocentric 3D Visual Span](https://mlsanity.com/paper/2511.18470/) (2025)
- [Implicit Search Intent Recognition using EEG and Eye Tracking: Novel Dataset and Cross-User Prediction](https://mlsanity.com/paper/2508.01860/) (2025)
- [Gaze Prediction as a Function of Eye Movement Type and Individual Differences](https://mlsanity.com/paper/2501.00597/) (2024)

## Abstract (arXiv)

Visual attention is highly fragmented during mobile interactions, but the erratic nature of attention shifts currently limits attentive user interfaces to adapting after the fact, i.e. after shifts have already happened. We instead study attention forecasting -- the challenging task of predicting users' gaze behaviour (overt visual attention) in the near future. We present a novel long-term dataset of everyday mobile phone interactions, continuously recorded from 20 participants engaged in common activities on a university campus over 4.5 hours each (more than 90 hours in total). We propose a proof-of-concept method that uses device-integrated sensors and body-worn cameras to encode rich information on device usage and users' visual scene. We demonstrate that our method can forecast bidirectional attention shifts and predict whether the primary attentional focus is on the handheld mobile device. We study the impact of different feature sets on performance and discuss the significant potential but also remaining challenges of forecasting user attention during mobile interactions.