# Human Object Interaction Detection using Two-Direction Spatial Enhancement and Exclusive Object Prior

arXiv: https://arxiv.org/abs/2105.03089 · HTML: https://mlsanity.com/paper/2105.03089/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 7 May 2021.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [R-FCN: Object Detection via Region-based Fully Convolutional Networks](https://mlsanity.com/paper/1605.06409/) (2016)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [Detecting and Recognizing Human-Object Interactions](https://mlsanity.com/paper/1704.07333/) (2017)
- [Visual Semantic Role Labeling](https://mlsanity.com/paper/1505.04474/) (2015)
- [Learning Human-Object Interactions by Graph Parsing Neural Networks](https://mlsanity.com/paper/1808.07962/) (2018)
- [iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection](https://mlsanity.com/paper/1808.10437/) (2018)
- [Visual Translation Embedding Network for Visual Relation Detection](https://mlsanity.com/paper/1702.08319/) (2017)
- [Pose-aware Multi-level Feature Network for Human Object Interaction Detection](https://mlsanity.com/paper/1909.08453/) (2019)
- [Attentional Pooling for Action Recognition](https://mlsanity.com/paper/1711.01467/) (2017)
- [Deep Contextual Attention for Human-Object Interaction Detection](https://mlsanity.com/paper/1910.07721/) (2019)
- [Contextual Action Recognition with R*CNN](https://mlsanity.com/paper/1505.01197/) (2015)
- [Pairwise Body-Part Attention for Recognizing Human-Object Interactions](https://mlsanity.com/paper/1807.10889/) (2018)
- [Large-Scale Visual Relationship Understanding](https://mlsanity.com/paper/1804.10660/) (2018)
- [On Exploring Undetermined Relationships for Visual Relationship Detection](https://mlsanity.com/paper/1905.01595/) (2019)
- [Detecting Visual Relationships Using Box Attention](https://mlsanity.com/paper/1807.02136/) (2018)
- [Single Image Action Recognition using Semantic Body Part Actions](https://mlsanity.com/paper/1612.04520/) (2016)
- [Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion](https://mlsanity.com/paper/2006.02176/) (2020)
- [Actions and Attributes from Wholes and Parts](https://mlsanity.com/paper/1412.2604/) (2014)
- [No-Frills Human-Object Interaction Detection: Factorization, Layout Encodings, and Training Techniques](https://mlsanity.com/paper/1811.05967/) (2018)
- [Loss Guided Activation for Action Recognition in Still Images](https://mlsanity.com/paper/1812.04194/) (2018)

## Built on by (most cited first, 3 of 3)

- [StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset](https://mlsanity.com/paper/2407.20545/) (2024)
- [Geometric Features Enhanced Human-Object Interaction Detection](https://mlsanity.com/paper/2406.18691/) (2024)
- [HOKEM: Human and Object Keypoint-based Extension Module for Human-Object Interaction Detection](https://mlsanity.com/paper/2306.14260/) (2023)

## Abstract (arXiv)

Human-Object Interaction (HOI) detection aims to detect visual relations between human and objects in images. One significant problem of HOI detection is that non-interactive human-object pair can be easily mis-grouped and misclassified as an action, especially when humans are close and performing similar actions in the scene. To address the mis-grouping problem, we propose a spatial enhancement approach to enforce fine-level spatial constraints in two directions from human body parts to the object center, and from object parts to the human center. At inference, we propose a human-object regrouping approach by considering the object-exclusive property of an action, where the target object should not be shared by more than one human. By suppressing non-interactive pairs, our approach can decrease the false positives. Experiments on V-COCO and HICO-DET datasets demonstrate our approach is more robust compared to the existing methods under the presence of multiple humans and objects in the scene.