mlsanityData to 7 Oct 2026

Learning to Detect Human-Object Interactions

arXiv 1702.05448 ↗First posted 17 Feb 2017cs.CVcited by 238

Not on the map. Why?

Cited by 21 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the problem of detecting human-object interactions (HOI) in static images, defined as predicting a human and an object bounding box with an interaction class label that connects them. HOI detection is a fundamental problem in computer vision as it provides semantic information about the interactions among the detected objects. We introduce HICO-DET, a new large benchmark for HOI detection, by augmenting the current HICO classification benchmark with instance annotations. To solve the task, we propose Human-Object Region-based Convolutional Neural Networks (HO-RCNN). At the core of our HO-RCNN is the Interaction Pattern, a novel DNN input that characterizes the spatial relations between two bounding boxes. Experiments on HICO-DET demonstrate that our HO-RCNN, by exploiting human-object spatial relations through Interaction Patterns, significantly improves the performance of HOI detection over baseline approaches.

Key facts. Cited by 238 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 36; 2022: 44; 2023: 50; 2024: 32; 2025: 49; 2026: 27.

238 papers.

Sort:

By area:

  1. 2023Reconstructing Hands in 3D with Transformerscited by 298
  2. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  3. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  4. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  5. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
  6. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  7. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  8. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
  9. 2022Panoptic Scene Graph Generationcited by 114
  10. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  11. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  12. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
38 more
  1. 2023OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mappingcited by 83
  2. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  3. 2021Detecting Human-Object Interaction via Fabricated Compositional Learningcited by 79
  4. 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
  5. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  6. 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 60
  7. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  8. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  9. 2022Disentangling Visual Embeddings for Attributes and Objectscited by 55
  10. 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
  11. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  12. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 46
  13. 2022Webly Supervised Concept Expansion for General Purpose Vision Modelscited by 45
  14. 2022CholecTriplet2021: A benchmark challenge for surgical action triplet recognitioncited by 44
  15. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 44
  16. 2023ViPLO: Vision Transformer based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detectioncited by 41
  17. 2023TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoningcited by 41
  18. 2022Interactiveness Field in Human-Object Interactionscited by 40
  19. 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
  20. 2023Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memorycited by 38
  21. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  22. 2022Interaction Region Visual Transformer for Egocentric Action Anticipationcited by 36
  23. 2022What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactionscited by 35
  24. 2023Relational Context Learning for Human-Object Interaction Detectioncited by 35
  25. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  26. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
  27. 2023Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Modelscited by 34
  28. 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
  29. 2023UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detectioncited by 31
  30. 2022End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillationcited by 30
  31. 2023FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detectioncited by 30
  32. 2021QAHOI: Query-Based Anchors for Human-Object Interaction Detectioncited by 28
  33. 2022HAKE: A Knowledge Engine Foundation for Human Activity Understandingcited by 28
  34. 2024Open-World Human-Object Interaction Detection via Multi-modal Promptscited by 28
  35. 2024EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detectioncited by 28
  36. 2023Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Modelscited by 27
  37. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  38. 2024Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detectioncited by 26

50 most cited of 238 All 238 (JSON)