mlsanityData to 14 Oct 2026

A Review of Human-Object Interaction Detection

arXiv 2408.10641 ↗First posted 20 Aug 2024cs.CV cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Human-object interaction (HOI) detection plays a key role in high-level visual understanding, facilitating a deep comprehension of human activities. Specifically, HOI detection aims to locate the humans and objects involved in interactions within images or videos and classify the specific interactions between them. The success of this task is influenced by several key factors, including the accurate localization of human and object instances, as well as the correct classification of object categories and interaction relationships. This paper systematically summarizes and discusses the recent work in image-based HOI detection. First, the mainstream datasets involved in HOI relationship detection are introduced. Furthermore, starting with two-stage methods and end-to-end one-stage detection approaches, this paper comprehensively discusses the current developments in image-based HOI detection, analyzing the strengths and weaknesses of these two methods. Additionally, the advancements of zero-shot learning, weakly supervised learning, and the application of large-scale language models in HOI detection are discussed. Finally, the current challenges in HOI detection are outlined, and potential research directions and future trends are explored.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 59 of its 74 references that are in the map. First posted on arXiv 20 Aug 2024.

Builds on

59 of 74 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2015Fast R-CNNcited by 2,933
  5. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  6. 2017Learning to Detect Human-Object Interactionscited by 240
  7. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  8. 2015Visual Semantic Role Labelingcited by 190
  9. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  10. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  11. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  12. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
38 more
  1. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 135
  2. 2020Visual Compositional Learning for Human-Object Interaction Detectioncited by 130
  3. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 127
  4. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 120
  5. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  6. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  7. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  8. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  9. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  10. 2020Detailed 2D-3D Joint Representation for Human-Object Interactioncited by 93
  11. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  12. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  13. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  14. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  15. 2021Detecting Human-Object Interaction via Fabricated Compositional Learningcited by 79
  16. 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
  17. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  18. 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 61
  19. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  20. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  21. 2020Contextual Heterogeneous Graph Network for Human-Object Interaction Detectioncited by 50
  22. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 47
  23. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 45
  24. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  25. 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
  26. 2017Weakly-supervised learning of visual relationscited by 36
  27. 2023Relational Context Learning for Human-Object Interaction Detectioncited by 36
  28. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  29. 2020DIRV: Dense Interaction Region Voting for End-to-End Human-Object Interaction Detectioncited by 31
  30. 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
  31. 2023UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detectioncited by 31
  32. 2023FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detectioncited by 30
  33. 2017PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCNcited by 25
  34. 2021Exploiting Scene Graphs for Human-Object Interaction Detectioncited by 22
  35. 2022Category-Aware Transformer Network for Better Human-Object Interaction Detectioncited by 22
  36. 2022Iwin: Human-Object Interaction Detection via Transformer with Irregular Windowscited by 19
  37. 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
  38. 2020Explanation-based Weakly-supervised Learning of Visual Relations with Graph Networkscited by 13

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 5.

5 papers.

  1. 2025ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modelingcited by 12
  2. 2025MHR: Momentum Human Rigcited by 9
  3. 2025Human Body Restoration with One-Step Diffusion Model and A New Benchmarkcited by 5
  4. 2025HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidancecited by 5
  5. 2025Prompt Guidance and Human Proximal Perception for HOT Prediction with Regional Joint Losscited by 1