mlsanityData to 7 Oct 2026

Videos as Space-Time Region Graphs

arXiv 1806.01810 ↗First posted 5 Jun 2018cs.CVcited by 221

Not on the map. Why?

Abstract (arXiv)

How do humans recognize the action "opening a book" ? We argue that there are two important cues: modeling temporal shape dynamics and modeling functional relationships between humans and objects. In this paper, we propose to represent videos as space-time region graphs which capture these two important cues. Our graph nodes are defined by the object region proposals from different frames in a long range video. These nodes are connected by two types of relations: (i) similarity relations capturing the long range dependencies between correlated objects and (ii) spatial-temporal relations capturing the interactions between nearby objects. We perform reasoning on this graph representation via Graph Convolutional Networks. We achieve state-of-the-art results on both Charades and Something-Something datasets. Especially for Charades, we obtain a huge 4.4% gain when our model is applied in complex environments.

Key facts. Cited by 221 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 85; 2022: 52; 2023: 44; 2024: 26; 2025: 7; 2026: 7.

221 papers.

Sort:

By area:

  1. 2021Multiscale Vision Transformerscited by 896
  2. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  3. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  4. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  5. 2021Towards Long-Form Video Understandingcited by 146
  6. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  7. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  8. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  9. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  10. 2021VidTr: Video Transformer Without Convolutionscited by 112
  11. 2021Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayerscited by 104
  12. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
38 more
  1. 2023Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detectioncited by 88
  2. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  3. 2021Adaptive Focus for Efficient Video Recognitioncited by 76
  4. 2021Object-Region Video Transformerscited by 73
  5. 2023Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Modelscited by 72
  6. 2022Teaching Structured Vision&Language Concepts to Vision&Language Modelscited by 71
  7. 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
  8. 2021Temporal Query Networks for Fine-grained Video Understandingcited by 68
  9. 2022Invariant Grounding for Video Question Answeringcited by 68
  10. 2021Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identificationcited by 63
  11. 2022Video Graph Transformer for Video Question Answeringcited by 63
  12. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  13. 2023Action Scene Graphs for Long-Form Understanding of Egocentric Videoscited by 51
  14. 2021Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixingcited by 49
  15. 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
  16. 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
  17. 2021Forecasting Action through Contact Representations from First Person Videocited by 46
  18. 2021Spatial Dual-Modality Graph Reasoning for Key Information Extractioncited by 43
  19. 2021Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentationcited by 43
  20. 2023Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphscited by 39
  21. 2021Unified Graph Structured Models for Video Understandingcited by 38
  22. 2022Latent Graph Representations for Critical View of Safety Assessmentcited by 37
  23. 2022Interaction Region Visual Transformer for Egocentric Action Anticipationcited by 36
  24. 2023Video-Mined Task Graphs for Keystep Recognition in Instructional Videoscited by 35
  25. 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
  26. 2023Sheaf Hypergraph Networkscited by 33
  27. 2023On the Benefits of 3D Pose and Tracking for Human Action Recognitioncited by 32
  28. 2021Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognitioncited by 31
  29. 2021TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learningcited by 31
  30. 2023Helping Hands: An Object-Aware Ego-Centric Video Recognition Modelcited by 31
  31. 2021Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answeringcited by 29
  32. 2021Graph Convolutional Module for Temporal Action Localization in Videoscited by 29
  33. 2022(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answeringcited by 29
  34. 2023A Review of Deep Learning for Video Captioningcited by 28
  35. 2023Multimodal Distillation for Egocentric Action Recognitioncited by 28
  36. 2021CT-Net: Channel Tensorization Network for Video Classificationcited by 26
  37. 2021No frame left behind: Full Video Action Recognitioncited by 24
  38. 2022Is an Object-Centric Video Representation Beneficial for Transfer?cited by 24

50 most cited of 221 All 221 (JSON)