mlsanityData to 14 Oct 2026

Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos

arXiv 2309.05943 ↗First posted 12 Sep 2023cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

This work focuses on anticipating long-term human actions, particularly using short video segments, which can speed up editing workflows through improved suggestions while fostering creativity by suggesting narratives. To this end, we imbue a transformer network with a symbolic knowledge graph for action anticipation in video segments by boosting certain aspects of the transformer's attention mechanism at run-time. Demonstrated on two benchmark datasets, Breakfast and 50Salads, our approach outperforms current state-of-the-art methods for long-term action anticipation using short video context by up to 9%.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

29 of 42 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2016Layer Normalizationcited by 5,669
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  7. 2020Concept Bottleneck Modelscited by 1,032
  8. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  9. 2020Rescaling Egocentric Visioncited by 540
  10. 2021Anticipative Video Transformercited by 200
  11. 2021Towards Long-Form Video Understandingcited by 146
  12. 2018When will you do what? - Anticipating Temporal Occurrences of Activitiescited by 107
17 more
  1. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 105
  2. 2020A Comprehensive Study of Deep Video Action Recognitioncited by 88
  3. 2016The More You Know: Using Knowledge Graphs for Image Classificationcited by 63
  4. 2022Video Transformers: A Surveycited by 63
  5. 2022Future Transformer for Long-term Action Anticipationcited by 58
  6. 2018Zero-Shot Anticipation for Instructional Activitiescited by 43
  7. 2019Predicting the Future: A Jointly Learnt Model for Action Anticipationcited by 41
  8. 2020Long-Term Anticipation of Activities with Cycle Consistencycited by 40
  9. 2019Uncertainty-Aware Anticipation of Activitiescited by 39
  10. 2017Encouraging LSTMs to Anticipate Actions Very Earlycited by 37
  11. 2021Anticipating human actions by correlating past with the future with Jaccard similarity measurescited by 36
  12. 2020What can I do here? A Theory of Affordances in Reinforcement Learningcited by 35
  13. 2022FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relationscited by 20
  14. 2020Video Action Understandingcited by 13
  15. 2023Sample-Efficient Learning of Novel Visual Conceptscited by 8
  16. 2022Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipationcited by 6
  17. 2020Visual Concept Reasoning Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generationcited by 14
  2. 2024ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decompositioncited by 14
  3. 2024HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipationcited by 3
  4. 2026Neuro-Symbolic Hierarchical Intention Anticipation in Human Behaviorcited by 0