mlsanityData to 7 Oct 2026

SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videos

arXiv 2109.00829 ↗First posted 2 Sep 2021cs.CV cs.AI cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Action anticipation in egocentric videos is a difficult task due to the inherently multi-modal nature of human actions. Additionally, some actions happen faster or slower than others depending on the actor or surrounding context which could vary each time and lead to different predictions. Based on this idea, we build upon RULSTM architecture, which is specifically designed for anticipating human actions, and propose a novel attention-based technique to evaluate, simultaneously, slow and fast features extracted from three different modalities, namely RGB, optical flow, and extracted objects. Two branches process information at different time scales, i.e., frame-rates, and several fusion schemes are considered to improve prediction accuracy. We perform extensive experiments on EpicKitchens-55 and EGTEA Gaze+ datasets, and demonstrate that our technique systematically improves the results of RULSTM architecture for Top-5 accuracy metric at different anticipation times.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

25 of 38 references.

References · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  3. 2018SlowFast Networks for Video Recognitioncited by 1,862
  4. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  5. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  6. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  7. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  8. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
  9. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  10. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  11. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  12. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
13 more
  1. 2020Multi-Modal Domain Adaptation for Fine-Grained Action Recognitioncited by 115
  2. 2019Peeking into the Future: Predicting Future Person Activities and Locations in Videoscited by 112
  3. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 106
  4. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 77
  5. 2019Next-Active-Object prediction from Egocentric Videoscited by 55
  6. 2016First-Person Activity Forecasting with Online Inverse Reinforcement Learningcited by 40
  7. 2020Symbiotic Attention with Privileged Information for Egocentric Action Recognitioncited by 27
  8. 2020Knowledge Distillation for Action Anticipation via Label Smoothingcited by 21
  9. 2020Modality Compensation Network: Cross-Modal Adaptation for Action Recognitioncited by 16
  10. 2017Visual Forecasting by Imitating Dynamics in Natural Sequencescited by 14
  11. 2018Anticipation in Human-Robot Cooperation: A Recurrent Neural Network Approach for Multiple Action Sequences Predictioncited by 12
  12. 2017Forecasting Hands and Objects in Future Framescited by 3
  13. 2017Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 5; 2024: 4; 2025: 3; 2026: 2.

20 papers.

Sort:

  1. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  2. 2024Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Modelscited by 32
  3. 2022Rethinking Learning Approaches for Long-Term Action Anticipationcited by 29
  4. 2023StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipationcited by 23
  5. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  6. 2025Multimodal Large Models Are Effective Action Anticipatorscited by 10
  7. 2022TAMFormer: Multi-Modal Transformer with Learned Attention Mask for Early Intent Predictioncited by 9
  8. 2024Semantically Guided Representation Learning For Action Anticipationcited by 9
  9. 2024Human Action Anticipation: A Surveycited by 8
  10. 2022Unified Recurrence Modeling for Video Action Anticipationcited by 7
  11. 2023Privacy-preserving Early Detection of Epileptic Seizures in Videoscited by 7
  12. 2021Learning to Discriminate Information for Online Action Detection: Analysis and Applicationcited by 6
8 more
  1. 2023VS-TransGRU: A Novel Transformer-GRU-based Framework Enhanced by Visual-Semantic Fusion for Egocentric Action Anticipationcited by 4
  2. 2022Inductive Attention for Video Action Anticipationcited by 1
  3. 2023Early Action Recognition with Action Prototypescited by 1
  4. 2024Intention Action Anticipation Model with Guide-Feedback Loop Mechanismcited by 1
  5. 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaborationcited by 1
  6. 2025Action-Dynamics Modeling and Cross-Temporal Interaction for Online Action Understandingcited by 0
  7. 2025State-Change Learning for Prediction of Future Events in Endoscopic Videoscited by 0
  8. 2026Neuro-Symbolic Hierarchical Intention Anticipation in Human Behaviorcited by 0