mlsanityData to 7 Oct 2026

Human Action Anticipation: A Survey

arXiv 2410.14045 ↗First posted 17 Oct 2024cs.CV cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Predicting future human behavior is an increasingly popular topic in computer vision, driven by the interest in applications such as autonomous vehicles, digital assistants and human-robot interactions. The literature on behavior prediction spans various tasks, including action anticipation, activity forecasting, intent prediction, goal prediction, and so on. Our survey aims to tie together this fragmented literature, covering recent technical innovations as well as the development of new large-scale datasets for model training and evaluation. We also summarize the widely-used metrics for different tasks and provide a comprehensive performance comparison of existing approaches on eleven action anticipation datasets. This survey serves as not only a reference for contemporary methodologies in action anticipation, but also a guideline for future research direction of this evolving landscape.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 136 of its 193 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

136 of 193 references.

References · 136

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2023Segment Anythingcited by 8,158
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2017The Kinetics Human Action Video Datasetcited by 1,894
  9. 2018SlowFast Networks for Video Recognitioncited by 1,862
  10. 2016Generative Adversarial Imitation Learningcited by 1,576
  11. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  12. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
38 more
  1. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  2. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  3. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  4. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  5. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  6. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  7. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  8. 2018A Short Note about Kinetics-600cited by 331
  9. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
  10. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  11. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  12. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 232
  13. 2019Human Motion Trajectory Prediction: A Surveycited by 231
  14. 2018Human Action Recognition and Prediction: A Surveycited by 226
  15. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  16. 2021Anticipative Video Transformercited by 197
  17. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  18. 2019Video Representation Learning by Dense Predictive Codingcited by 175
  19. 2020Human Action Recognition from Various Data Modalities: A Reviewcited by 162
  20. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  21. 2018Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Imagescited by 118
  22. 2015Unsupervised Learning from Narrated Instruction Videoscited by 116
  23. 2020EGO-TOPO: Environment Affordances from Egocentric Videocited by 111
  24. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  25. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  26. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 106
  27. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 104
  28. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  29. 2015Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videoscited by 93
  30. 2019Procedure Planning in Instructional Videoscited by 92
  31. 2020A Short Note on the Kinetics-700-2020 Human Action Datasetcited by 89
  32. 2021Long Short-Term Transformer for Online Action Detectioncited by 87
  33. 2021OadTR: Online Action Detection with Transformerscited by 83
  34. 2019Oops! Predicting Unintentional Action in Videocited by 82
  35. 2018Temporal Recurrent Networks for Online Action Detectioncited by 79
  36. 2020The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domaincited by 79
  37. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  38. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 77

50 most cited of 136 All 136 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 1.

8 papers.

  1. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  2. 2025Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understandingcited by 18
  3. 2025Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video?cited by 3
  4. 2026Understanding Multimodal Complementarity for Single-Frame Action Anticipationcited by 2
  5. 2025Poly-Autoregressive Prediction for Modeling Interactionscited by 1
  6. 2025Countering Multi-modal Representation Collapse through Rank-targeted Fusioncited by 1
  7. 2025Seeing is Believing (and Predicting): Context-Aware Multi-Human Behavior Prediction with Vision Language Modelscited by 1
  8. 2025Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challengescited by 0