mlsanityData to 7 Oct 2026

Rethinking Learning Approaches for Long-Term Action Anticipation

arXiv 2210.11566 ↗First posted 20 Oct 2022cs.CV cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Action anticipation involves predicting future actions having observed the initial portion of a video. Typically, the observed video is processed as a whole to obtain a video-level representation of the ongoing activity in the video, which is then used for future prediction. We introduce ANTICIPATR which performs long-term action anticipation leveraging segment-level representations learned using individual segments from different activities, in addition to a video-level representation. We propose a two-stage learning approach to train a novel transformer-based model that uses these two types of representations to directly predict a set of future action instances over any given anticipation duration. Results on Breakfast, 50Salads, Epic-Kitchens-55, and EGTEA Gaze+ datasets demonstrate the effectiveness of our approach.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 52 of its 69 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

52 of 69 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2021ViViT: A Video Vision Transformercited by 1,610
  6. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  7. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  8. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  9. 2017On human motion prediction using recurrent neural networkscited by 336
  10. 2018Video Action Transformer Networkcited by 316
  11. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  12. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
38 more
  1. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  2. 2021Anticipative Video Transformercited by 197
  3. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  4. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  5. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  6. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  7. 2019Peeking into the Future: Predicting Future Person Activities and Locations in Videoscited by 112
  8. 2021VidTr: Video Transformer Without Convolutionscited by 112
  9. 2018TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agentscited by 111
  10. 2020EGO-TOPO: Environment Affordances from Egocentric Videocited by 111
  11. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 106
  12. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 104
  13. 2021MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Predictioncited by 100
  14. 2019Procedure Planning in Instructional Videoscited by 92
  15. 2018Timeception for Complex Action Recognitioncited by 89
  16. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 77
  17. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  18. 2017ActionVLAD: Learning spatio-temporal aggregation for action classificationcited by 73
  19. 2017Dual Motion GAN for Future-Flow Embedded Video Predictioncited by 69
  20. 2019VideoGraph: Recognizing Minutes-Long Human Activities in Videoscited by 57
  21. 2017Predicting Deeper into the Future of Semantic Segmentationcited by 51
  22. 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
  23. 2018Zero-Shot Anticipation for Instructional Activitiescited by 43
  24. 2020Long-Term Anticipation of Activities with Cycle Consistencycited by 39
  25. 2019Uncertainty-Aware Anticipation of Activitiescited by 38
  26. 2021Self-Regulated Learning for Egocentric Video Activity Anticipationcited by 32
  27. 2019A Variational Auto-Encoder Model for Stochastic Point Processescited by 30
  28. 2019Relational Action Forecastingcited by 27
  29. 2021Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videoscited by 23
  30. 2019Action Anticipation with RBF Kernelized Feature Mapping RNNcited by 20
  31. 2021SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videoscited by 20
  32. 2019Forecasting future action sequences with attention: a new approach to weakly supervised action forecastingcited by 19
  33. 2020Adversarial Generative Grammars for Human Activity Predictioncited by 19
  34. 2018Action Anticipation By Predicting Future Dynamic Imagescited by 18
  35. 2019Forecasting Future Action Sequences with Neural Memory Networkscited by 18
  36. 2018Online Detection of Action Start in Untrimmed, Streaming Videoscited by 17
  37. 2018Bayesian Prediction of Future Street Scenes using Synthetic Likelihoodscited by 17
  38. 2017Visual Forecasting by Imitating Dynamics in Natural Sequencescited by 14

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 10; 2025: 10; 2026: 2.

29 papers.

Sort:

By area:

  1. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  2. 2022Interaction Region Visual Transformer for Egocentric Action Anticipationcited by 36
  3. 2023PALM: Predicting Actions through Language Modelscited by 28
  4. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  5. 2024Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videoscited by 15
  6. 2024AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipationcited by 14
  7. 2023Action Anticipation with Goal Consistencycited by 12
  8. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  9. 2024PEAR: Phrase-Based Hand-Object Interaction Anticipationcited by 10
  10. 2023GePSAn: Generative Procedure Step Anticipation in Cooking Videoscited by 9
  11. 2023DiffAnt: Diffusion Models for Action Anticipationcited by 9
  12. 2025Action Anticipation from SoccerNet Football Video Broadcastscited by 9
17 more
  1. 2024Human Action Anticipation: A Surveycited by 8
  2. 2024Uncertainty-boosted Robust Video Activity Anticipationcited by 7
  3. 2024Bidirectional Progressive Transformer for Interaction Intention Anticipationcited by 6
  4. 2024Gated Temporal Diffusion for Stochastic Long-Term Dense Anticipationcited by 6
  5. 2024Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipationcited by 6
  6. 2025CLOT: Closed Loop Optimal Transport for Unsupervised Action Segmentationcited by 6
  7. 2025Vision and Intention Boost Large Language Model in Long-Term Action Anticipationcited by 5
  8. 2025Enhancing Visual Planning with Auxiliary Tasks and Multi-token Predictioncited by 5
  9. 2024ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipationcited by 4
  10. 2025MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Anticipationcited by 3
  11. 2025CLAD: Constrained Latent Action Diffusion for Vision-Language Procedure Planningcited by 2
  12. 2025Bidirectional Action Sequence Learning for Long-term Action Anticipation with Large Language Modelscited by 2
  13. 2024Intention Action Anticipation Model with Guide-Feedback Loop Mechanismcited by 1
  14. 2025FDSG: Forecasting Dynamic Scene Graphscited by 1
  15. 2025MixANT: Observation-dependent Memory Propagation for Stochastic Dense Action Anticipationcited by 1
  16. 2026Integrating Affordances and Attention models for Short-Term Object Interaction Anticipationcited by 1
  17. 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaborationcited by 1