VideoGraph: Recognizing Minutes-Long Human Activities in Videos
Not on the map. Why?
Abstract (arXiv)
Many human activities take minutes to unfold. To represent them, related works opt for statistical pooling, which neglects the temporal structure. Others opt for convolutional methods, as CNN and Non-Local. While successful in learning temporal concepts, they are short of modeling minutes-long temporal dependencies. We propose VideoGraph, a method to achieve the best of two worlds: represent minutes-long human activities and learn their underlying temporal structure. VideoGraph learns a graph-based representation for human activities. The graph, its nodes and edges are learned entirely from video datasets, making VideoGraph applicable to problems without node-level annotation. The result is improvements over related works on benchmarks: Epic-Kitchen and Breakfast. Besides, we demonstrate that VideoGraph is able to learn the temporal structure of human activities in minutes-long videos.
Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 May 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 10; 2023: 10; 2024: 15; 2025: 11; 2026: 3.
57 papers.
Sort:
By area:
- 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
- 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
- 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
- 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
- 2022Learning To Recognize Procedural Activities with Distant Supervisioncited by 85
- 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
- 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
- 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
- 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
38 more
- 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
- 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
- 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understandingcited by 30
- 2022Rethinking Learning Approaches for Long-Term Action Anticipationcited by 29
- 2024VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMscited by 29
- 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
- 2023PALM: Predicting Actions through Language Modelscited by 28
- 2021Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activitiescited by 26
- 2021No frame left behind: Full Video Action Recognitioncited by 24
- 2022C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentationcited by 23
- 2024Video Token Merging for Long-form Video Understandingcited by 22
- 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understandingcited by 17
- 2025BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimationcited by 17
- 2022Gate-Shift-Fuse for Video Action Recognitioncited by 15
- 2021Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognitioncited by 14
- 2022Turbo Training with Token Dropoutcited by 14
- 2024Enhancing Long Video Understanding via Hierarchical Event-Based Memorycited by 14
- 2022Concept Graph Neural Networks for Surgical Video Understandingcited by 13
- 2023Chop & Learn: Recognizing and Generating Object-State Compositionscited by 13
- 2023United We Stand, Divided We Fall: UnityGraph for Unsupervised Procedure Learning from Videoscited by 13
- 2021Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videoscited by 11
- 2024IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videoscited by 11
- 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipationcited by 10
- 2023DiffAnt: Diffusion Models for Action Anticipationcited by 9
- 2021PGT: A Progressive Method for Training Models on Long Videoscited by 7
- 2023Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understandingcited by 7
- 2024AdaCM2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reductioncited by 7
- 2023Are current long-term video understanding datasets long-term?cited by 6
- 2024Towards Long Video Understanding via Fine-detailed Video Story Generationcited by 6
- 2024Gaze-Guided Graph Neural Network for Action Anticipation Conditioned on Intentioncited by 5
- 2025Vision and Intention Boost Large Language Model in Long-Term Action Anticipationcited by 5
- 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognitioncited by 4
- 2024A Transformer-Based Model for the Prediction of Human Gaze Behavior on Videoscited by 3
- 2025VideoMAP: Toward Scalable Mamba-based Video Autoregressive Pretrainingcited by 3
- 2025LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoderscited by 3
- 2022SVGraph: Learning Semantic Graphs from Instructional Videoscited by 2
- 2023Video BagNet: short temporal receptive fields increase robustness in long-term action recognitioncited by 2
- 2025REEF: Relevance-Aware and Efficient LLM Adapter for Video Understandingcited by 2
50 most cited of 57 All 57 (JSON)
- 2026VidParse: Online Parsing of Egocentric Procedures Like a Pro27 Aug 2026
- 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaboration10 Jun 2026
- 2026Graph-to-Frame RAG: Visual-Space Knowledge Fusion for Training-Free and Auditable Video Reasoning6 Apr 2026
- 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding15 Oct 2025
- 2025Perceive, Verify and Understand Long Video: Multi-Granular Perception and Active Verification via Interactive Agents29 Sep 2025
- 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation3 Aug 2025
- 2025Cross-Modal Dual-Causal Learning for Long-Term Action Recognition9 Jul 2025
- 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition29 Jun 2025
- 2025Vision and Intention Boost Large Language Model in Long-Term Action Anticipation3 May 2025
- 2025REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding7 Apr 2025
- 2025LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoders4 Apr 2025
- 2025BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimation26 Mar 2025
38 more
- 2025VideoMAP: Toward Scalable Mamba-based Video Autoregressive Pretraining16 Mar 2025
- 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding11 Mar 2025
- 2024Towards Long Video Understanding via Fine-detailed Video Story Generation9 Dec 2024
- 2024AdaCM2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reduction19 Nov 2024
- 2024IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos18 Nov 2024
- 2024Video Token Merging for Long-form Video Understanding31 Oct 2024
- 2024VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs30 Sep 2024
- 2024AMEGO: Active Memory from long EGOcentric videos17 Sep 2024
- 2024Enhancing Long Video Understanding via Hierarchical Event-Based Memory10 Sep 2024
- 2024Box2Flow: Instance-based Action Flow Graphs from Videos30 Aug 2024
- 2024DrVideo: Document Retrieval Based Long Video Understanding18 Jun 2024
- 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges24 Apr 2024
- 2024A Transformer-Based Model for the Prediction of Human Gaze Behavior on Videos10 Apr 2024
- 2024Gaze-Guided Graph Neural Network for Action Anticipation Conditioned on Intention10 Apr 2024
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr 2024
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent15 Mar 2024
- 2024VideoMamba: State Space Model for Efficient Video Understanding11 Mar 2024
- 2023A Simple LLM Framework for Long-Range Video Question-Answering28 Dec 2023
- 2023PALM: Predicting Actions through Language Models29 Nov 2023
- 2023DiffAnt: Diffusion Models for Action Anticipation27 Nov 2023
- 2023United We Stand, Divided We Fall: UnityGraph for Unsupervised Procedure Learning from Videos6 Nov 2023
- 2023Chop & Learn: Recognizing and Generating Object-State Compositions25 Sep 2023
- 2023Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understanding20 Sep 2023
- 2023Video BagNet: short temporal receptive fields increase robustness in long-term action recognition22 Aug 2023
- 2023Are current long-term video understanding datasets long-term?22 Aug 2023
- 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?31 Jul 2023
- 2023Selective Structured State-Spaces for Long-Form Video Understanding25 Mar 2023
- 2022C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation20 Dec 2022
- 2022Rethinking Learning Approaches for Long-Term Action Anticipation20 Oct 2022
- 2022Temporal Action Segmentation: An Analysis of Modern Techniques19 Oct 2022
- 2022Turbo Training with Token Dropout10 Oct 2022
- 2022SVGraph: Learning Semantic Graphs from Instructional Videos16 Jul 2022
- 2022Long Movie Clip Classification with State-Space Video Models4 Apr 2022
- 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos26 Mar 2022
- 2022Gate-Shift-Fuse for Video Action Recognition16 Mar 2022
- 2022Concept Graph Neural Networks for Surgical Video Understanding27 Feb 2022
- 2022Learning To Recognize Procedural Activities with Distant Supervision26 Jan 2022
- 2021Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videos12 Oct 2021
50 newest of 57 All 57 (JSON)
- 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
- 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
- 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
- 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognitioncited by 4
- 2025VideoMAP: Toward Scalable Mamba-based Video Autoregressive Pretrainingcited by 3
- 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition29 Jun 2025
- 2025VideoMAP: Toward Scalable Mamba-based Video Autoregressive Pretraining16 Mar 2025
- 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges24 Apr 2024
- 2024VideoMamba: State Space Model for Efficient Video Understanding11 Mar 2024
- 2023Selective Structured State-Spaces for Long-Form Video Understanding25 Mar 2023
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
- 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understandingcited by 17
- 2025REEF: Relevance-Aware and Efficient LLM Adapter for Video Understandingcited by 2
- 2026Graph-to-Frame RAG: Visual-Space Knowledge Fusion for Training-Free and Auditable Video Reasoningcited by 2
- 2026Graph-to-Frame RAG: Visual-Space Knowledge Fusion for Training-Free and Auditable Video Reasoning6 Apr 2026
- 2025REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding7 Apr 2025
- 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding11 Mar 2025
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr 2024
- 2023DiffAnt: Diffusion Models for Action Anticipationcited by 9
- 2023DiffAnt: Diffusion Models for Action Anticipation27 Nov 2023
- 2024Video Token Merging for Long-form Video Understandingcited by 22
- 2024Video Token Merging for Long-form Video Understanding31 Oct 2024
- 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
- 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
- 2022Learning To Recognize Procedural Activities with Distant Supervisioncited by 85
- 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
- 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
- 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
- 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
- 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
- 2022Rethinking Learning Approaches for Long-Term Action Anticipationcited by 29
- 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
- 2023PALM: Predicting Actions through Language Modelscited by 28
12 most cited of 42 All 42 (JSON)
- 2026VidParse: Online Parsing of Egocentric Procedures Like a Pro27 Aug 2026
- 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaboration10 Jun 2026
- 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation3 Aug 2025
- 2025Cross-Modal Dual-Causal Learning for Long-Term Action Recognition9 Jul 2025
- 2025Vision and Intention Boost Large Language Model in Long-Term Action Anticipation3 May 2025
- 2025LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoders4 Apr 2025
- 2025BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimation26 Mar 2025
- 2024Towards Long Video Understanding via Fine-detailed Video Story Generation9 Dec 2024
- 2024AdaCM2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reduction19 Nov 2024
- 2024IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos18 Nov 2024
- 2024AMEGO: Active Memory from long EGOcentric videos17 Sep 2024
- 2024Enhancing Long Video Understanding via Hierarchical Event-Based Memory10 Sep 2024
12 newest of 42 All 42 (JSON)