# Action Anticipation: Reading the Intentions of Humans and Robots

arXiv: https://arxiv.org/abs/1802.02788 · HTML: https://mlsanity.com/paper/1802.02788/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2018.

## Built on by (most cited first, 17 of 17)

- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [GIMO: Gaze-Informed Human Motion Prediction in Context](https://mlsanity.com/paper/2204.09443/) (2022)
- [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025)
- [Pose Forecasting in Industrial Human-Robot Collaboration](https://mlsanity.com/paper/2208.07308/) (2022)
- [Challenges and Trends in Egocentric Vision: A Survey](https://mlsanity.com/paper/2503.15275/) (2025)
- [Gaze-based intention estimation: principles, methodologies, and applications in HRI](https://mlsanity.com/paper/2302.04530/) (2023)
- [Toward Reliable Human Pose Forecasting with Uncertainty](https://mlsanity.com/paper/2304.06707/) (2023)
- [Best Practices for 2-Body Pose Forecasting](https://mlsanity.com/paper/2304.05758/) (2023)
- [Pose2Gaze: Eye-body Coordination during Daily Activities for Gaze Prediction from Full-body Poses](https://mlsanity.com/paper/2312.12042/) (2023)
- [Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation](https://mlsanity.com/paper/2210.05991/) (2022)
- [GazeMotion: Gaze-guided Human Motion Forecasting](https://mlsanity.com/paper/2403.09885/) (2024)
- [HaHeAE: Learning Generalisable Joint Representations of Human Hand and Head Movements in Extended Reality](https://mlsanity.com/paper/2410.16430/) (2024)
- [CASPER: Cognitive Architecture for Social Perception and Engagement in Robots](https://mlsanity.com/paper/2209.01012/) (2022)
- [HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario](https://mlsanity.com/paper/2509.10096/) (2025)
- [Event-based Timestamp Image Encoding Network for Human Action Recognition and Anticipation](https://mlsanity.com/paper/2104.05145/) (2021)
- [Anticipation through Head Pose Estimation: a preliminary study](https://mlsanity.com/paper/2408.05516/) (2024)
- [Uncertainty Aware-Predictive Control Barrier Functions: Safer Human Robot Interaction through Probabilistic Motion Forecasting](https://mlsanity.com/paper/2508.20812/) (2025)

## Abstract (arXiv)

Humans have the fascinating capacity of processing non-verbal visual cues to understand and anticipate the actions of other humans. This "intention reading" ability is underpinned by shared motor-repertoires and action-models, which we use to interpret the intentions of others as if they were our own. We investigate how the different cues contribute to the legibility of human actions during interpersonal interactions. Our first contribution is a publicly available dataset with recordings of human body-motion and eye-gaze, acquired in an experimental scenario with an actor interacting with three subjects. From these data, we conducted a human study to analyse the importance of the different non-verbal cues for action perception. As our second contribution, we used the motion/gaze recordings to build a computational model describing the interaction between two persons. As a third contribution, we embedded this model in the controller of an iCub humanoid robot and conducted a second human study, in the same scenario with the robot as an actor, to validate the model's "intention reading" capability. Our results show that it is possible to model (non-verbal) signals exchanged by humans during interaction, and how to incorporate such a mechanism in robotic systems with the twin goal of : (i) being able to "read" human action intentions, and (ii) acting in a way that is legible by humans.