mlsanityData to 14 Oct 2026

Intention-Conditioned Long-Term Human Egocentric Action Forecasting

arXiv 2207.12080 ↗First posted 25 Jul 2022cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

To anticipate how a human would act in the future, it is essential to understand the human intention since it guides the human towards a certain goal. In this paper, we propose a hierarchical architecture which assumes a sequence of human action (low-level) can be driven from the human intention (high-level). Based on this, we deal with Long-Term Action Anticipation task in egocentric videos. Our framework first extracts two level of human information over the N observed videos human actions through a Hierarchical Multi-task MLP Mixer (H3M). Then, we condition the uncertainty of the future through an Intention-Conditioned Variational Auto-Encoder (I-CVAE) that generates K stable predictions of the next Z=20 actions that the observed human might perform. By leveraging human intention as high-level information, we claim that our model is able to anticipate more time-consistent actions in the long-term, thus improving the results over baseline methods in EGO4D Challenge. This work ranked first in both CVPR@2022 and ECVV@2022 EGO4D LTA Challenge by providing more plausible anticipated sequences, improving the anticipation of nouns and overall actions. Webpage: https://evm7.github.io/icvae-page/

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

25 of 35 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  8. 2017The Kinetics Human Action Video Datasetcited by 1,909
  9. 2018SlowFast Networks for Video Recognitioncited by 1,867
  10. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  12. 2019An Introduction to Variational Autoencoderscited by 1,000
13 more
  1. 2017Focal Loss for Dense Object Detectioncited by 752
  2. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  3. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  4. 2017Toward Controlled Generation of Textcited by 272
  5. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 225
  6. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 107
  7. 2017Future Person Localization in First-Person Videoscited by 58
  8. 2021LOKI: Long Term and Key Intentions for Trajectory Predictioncited by 57
  9. 2021Learning the Predictability of the Futurecited by 49
  10. 2019Uncertainty-Aware Anticipation of Activitiescited by 39
  11. 2019A Variational Auto-Encoder Model for Stochastic Point Processescited by 30
  12. 2019Forecasting future action sequences with attention: a new approach to weakly supervised action forecastingcited by 19
  13. 2022Video + CLIP Baseline for Ego4D Long-term Action Anticipationcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 9; 2025: 9; 2026: 5.

35 papers.

Sort:

By area:

  1. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
  2. 2023An Outlook into the Future of Egocentric Visioncited by 84
  3. 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computationcited by 55
  4. 2022Interaction Region Visual Transformer for Egocentric Action Anticipationcited by 37
  5. 2023PDPP: Projected Diffusion for Procedure Planning in Instructional Videoscited by 36
  6. 2024Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Modelscited by 33
  7. 2023PALM: Predicting Actions through Language Modelscited by 29
  8. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  9. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  10. 2023Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023cited by 16
  11. 2022Predicting the Next Action by Modeling the Abstract Goalcited by 14
  12. 2023Action Anticipation with Goal Consistencycited by 12
23 more
  1. 2023HOI4ABOT: Human-Object Interaction Anticipation for Human Intention Reading Collaborative roBOTscited by 12
  2. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  3. 2024TI-PREGO: Chain of Thought and In-Context Learning for Online Mistake Detection in PRocedural EGOcentric Videoscited by 10
  4. 2023DiffAnt: Diffusion Models for Action Anticipationcited by 9
  5. 2024A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectivescited by 9
  6. 2024Human Action Anticipation: A Surveycited by 9
  7. 2025Action Anticipation from SoccerNet Football Video Broadcastscited by 9
  8. 2024Bidirectional Progressive Transformer for Interaction Intention Anticipationcited by 6
  9. 2024Gated Temporal Diffusion for Stochastic Long-Term Dense Anticipationcited by 6
  10. 2025Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectivescited by 5
  11. 2025Enhancing Visual Planning with Auxiliary Tasks and Multi-token Predictioncited by 5
  12. 2023VS-TransGRU: A Novel Transformer-GRU-based Framework Enhanced by Visual-Semantic Fusion for Egocentric Action Anticipationcited by 4
  13. 2025MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Anticipationcited by 3
  14. 2025Zero-Shot Temporal Interaction Localization for Egocentric Videoscited by 2
  15. 2025Optimizing Multitask Industrial Processes with Predictive Action Guidancecited by 1
  16. 2025Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Modelscited by 1
  17. 2025MixANT: Observation-dependent Memory Propagation for Stochastic Dense Action Anticipationcited by 1
  18. 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaborationcited by 1
  19. 2026Compositional Benchmark Synthesis for Hierarchical Human Action Recognitioncited by 1
  20. 2025See It Before You Grab It: Deep Learning-based Action Anticipation in Basketballcited by 0
  21. 2026Neuro-Symbolic Hierarchical Intention Anticipation in Human Behaviorcited by 0
  22. 2026AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalizationcited by 0
  23. 2026Language-Augmented Video Action Anticipation: Design Fundamentals, Benchmarks, and Open Challengescited by 0