mlsanityData to 7 Oct 2026

From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos

arXiv 2407.00917 ↗First posted 1 Jul 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Video-based Human-Object Interaction (HOI) recognition explores the intricate dynamics between humans and objects, which are essential for a comprehensive understanding of human behavior and intentions. While previous work has made significant strides, effectively integrating geometric and visual features to model dynamic relationships between humans and objects in a graph framework remains a challenge. In this work, we propose a novel end-to-end category to scenery framework, CATS, starting by generating geometric features for various categories through graphs respectively, then fusing them with corresponding visual features. Subsequently, we construct a scenery interactive graph with these enhanced geometric-visual features as nodes to learn the relationships among human and object categories. This methodological advance facilitates a deeper, more structured comprehension of interactions, bridging category-specific insights with broad scenery dynamics. Our method demonstrates state-of-the-art performance on two pivotal HOI benchmarks, including the MPHOI-72 dataset for multi-person HOIs and the single-person HOI CAD-120 dataset.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

31 of 49 references.

References · 31

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  6. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  7. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  8. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  9. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  10. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 160
  11. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128
  12. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
19 more
  1. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  2. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  3. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  4. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  5. 2020Contextual Heterogeneous Graph Network for Human-Object Interaction Detectioncited by 50
  6. 2020L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networkscited by 48
  7. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 46
  8. 2023ViPLO: Vision Transformer based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detectioncited by 41
  9. 2019Explainable Video Action Reasoning via Prior Knowledge and State Transitionscited by 33
  10. 2023Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentationcited by 33
  11. 2019Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networkscited by 29
  12. 2020BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialoguescited by 22
  13. 2022Geometric Features Informed Multi-person Human-object Interaction Recognition in Videoscited by 17
  14. 2021Learning Asynchronous and Sparse Human-Object Interaction in Videoscited by 16
  15. 2016Learning Models for Actions and Person-Object Interactions with Transfer to Question Answeringcited by 15
  16. 2020LIGHTEN: Learning Interactions with Graph and Hierarchical TEmporal Networks for HOI in videoscited by 14
  17. 2014Actions and Attributes from Wholes and Partscited by 12
  18. 2021Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognitioncited by 12
  19. 2022Multiclass-SGCN: Sparse Graph-based Trajectory Prediction with Agent Class Embeddingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025ViTE: Virtual Graph Trajectory Expert Router for Pedestrian Trajectory Predictioncited by 4
  2. 2025Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videoscited by 3
  3. 2026ART: Adaptive Relational Transformer for Pedestrian Trajectory Prediction with Temporal-Aware Relationscited by 1
  4. 2026HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimationcited by 0