mlsanityData to 7 Oct 2026

Relational Context Learning for Human-Object Interaction Detection

arXiv 2304.04997 ↗First posted 11 Apr 2023cs.CV cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Recent state-of-the-art methods for HOI detection typically build on transformer architectures with two decoder branches, one for human-object pair detection and the other for interaction classification. Such disentangled transformers, however, may suffer from insufficient context exchange between the branches and lead to a lack of context information for relational reasoning, which is critical in discovering HOI instances. In this work, we propose the multiplex relation network (MUREN) that performs rich context exchange between three decoder branches using unary, pairwise, and ternary relations of human, object, and interaction tokens. The proposed method learns comprehensive relational contexts for discovering HOI instances, achieving state-of-the-art performance on two standard benchmarks for HOI detection, HICO-DET and V-COCO.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 11 Apr 2023.

Builds on

36 of 49 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  7. 2017Learning to Detect Human-Object Interactionscited by 238
  8. 2018Exploring Visual Relationship for Image Captioningcited by 215
  9. 2015Visual Semantic Role Labelingcited by 188
  10. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  11. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  12. 2019Image Captioning: Transforming Objects into Wordscited by 171
24 more
  1. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  2. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  3. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  4. 2020Visual Compositional Learning for Human-Object Interaction Detectioncited by 129
  5. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  6. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  7. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  8. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  9. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  10. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  11. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  12. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  13. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  14. 2020Contextual Heterogeneous Graph Network for Human-Object Interaction Detectioncited by 50
  15. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 46
  16. 2020ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detectioncited by 45
  17. 2018A Structured Model For Action Detectioncited by 43
  18. 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
  19. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  20. 2020DIRV: Dense Interaction Region Voting for End-to-End Human-Object Interaction Detectioncited by 31
  21. 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
  22. 2020Image-to-Image Retrieval by Learning Similarity between Scene Graphscited by 23
  23. 2020IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videoscited by 12
  24. 2021Zero-Shot Action Recognition from Diverse Object-Scene Compositionscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 11; 2026: 7.

35 papers.

Sort:

By area:

  1. 2023Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Modelscited by 34
  2. 2024Open-World Human-Object Interaction Detection via Multi-modal Promptscited by 28
  3. 2024EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detectioncited by 28
  4. 2024Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detectioncited by 26
  5. 2024Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detectioncited by 25
  6. 2025Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understandingcited by 25
  7. 2023Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captionscited by 21
  8. 2024Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognitioncited by 17
  9. 2025Locality-Aware Zero-Shot Human-Object Interaction Detectioncited by 12
  10. 2025HOLa: Zero-Shot HOI Detection with Low-Rank Decomposed VLM Feature Adaptationcited by 12
  11. 2023Diagnosing Human-object Interaction Detectorscited by 11
  12. 2025Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibrationcited by 7
23 more
  1. 2026What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generationcited by 6
  2. 2024A Review of Human-Object Interaction Detectioncited by 5
  3. 2025CrossHOI-Bench: A Unified Benchmark for HOI Evaluation across Vision-Language Models and HOI-Specific Methodscited by 5
  4. 2023Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisorcited by 4
  5. 2023Enhancing HOI Detection with Contextual Cues from Large Vision-Language Modelscited by 3
  6. 2024Towards Zero-shot Human-Object Interaction Detection via Vision-Language Integrationcited by 3
  7. 2025Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videoscited by 3
  8. 2025Egocentric Human-Object Interaction Detection: A New Benchmark and Methodcited by 3
  9. 2025RoHOI: Robustness Benchmark for Human-Object Interaction Detectioncited by 3
  10. 2024Geometric Features Enhanced Human-Object Interaction Detectioncited by 2
  11. 2024Towards Flexible Visual Relationship Segmentationcited by 2
  12. 2024ContextHOI: Spatial Context Learning for Human-Object Interaction Detectioncited by 2
  13. 2024VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysiscited by 1
  14. 2025DQEN: Dual Query Enhancement Network for DETR-based HOI Detectioncited by 1
  15. 2026Mining Instance-Centric Vision-Language Contexts for Human-Object Interaction Detectioncited by 1
  16. 2026Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wildcited by 1
  17. 2024FreeA: Human-object Interaction Detection using Free Annotation Labelscited by 0
  18. 2025Contextualized Representation Learning for Effective Human-Object Interaction Detectioncited by 0
  19. 2025Part-Aware Bottom-Up Group Reasoning for Fine-Grained Social Interaction Detectioncited by 0
  20. 2026AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifierscited by 0
  21. 2026A Study of Failure Modes in Two-Stage Human-Object Interaction Detectioncited by 0
  22. 2026Beyond Isolated Entities: Relation-Aware Multi-Entity Modeling for Unsupervised Video Anomaly Detectioncited by 0
  23. 2026HEIR: Learning Human-Entity Interactions with Functional Rolescited by 0