# A Plug-and-Play Method for Rare Human-Object Interactions Detection by Bridging Domain Gap

arXiv: https://arxiv.org/abs/2407.21438 · HTML: https://mlsanity.com/paper/2407.21438/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [Visual Semantic Role Labeling](https://mlsanity.com/paper/1505.04474/) (2015)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information](https://mlsanity.com/paper/2103.05399/) (2021)
- [Visual Compositional Learning for Human-Object Interaction Detection](https://mlsanity.com/paper/2007.12407/) (2020)
- [GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection](https://mlsanity.com/paper/2203.13954/) (2022)
- [DRG: Dual Relation Graph for Human-Object Interaction Detection](https://mlsanity.com/paper/2008.11714/) (2020)
- [Mining the Benefits of Two-stage and One-stage HOI Detection](https://mlsanity.com/paper/2108.05077/) (2021)
- [Spatially Conditioned Graphs for Detecting Human-Object Interactions](https://mlsanity.com/paper/2012.06060/) (2020)
- [HOI Analysis: Integrating and Decomposing Human-Object Interaction](https://mlsanity.com/paper/2010.16219/) (2020)
- [Detecting Human-Object Interaction via Fabricated Compositional Learning](https://mlsanity.com/paper/2103.08214/) (2021)
- [Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detection](https://mlsanity.com/paper/2104.05269/) (2021)
- [RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection](https://mlsanity.com/paper/2209.01814/) (2022)
- [Detecting Human-Object Interactions via Functional Generalization](https://mlsanity.com/paper/1904.03181/) (2019)
- [Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers](https://mlsanity.com/paper/2107.12636/) (2021)
- [Interactiveness Field in Human-Object Interactions](https://mlsanity.com/paper/2204.07718/) (2022)
- [Human-Object Interaction Detection via Disentangled Transformer](https://mlsanity.com/paper/2204.09290/) (2022)
- [DA-DETR: Domain Adaptive Detection Transformer with Information Fusion](https://mlsanity.com/paper/2103.17084/) (2021)
- [Improving Transferability for Domain Adaptive Detection Transformers](https://mlsanity.com/paper/2204.14195/) (2022)
- [S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning](https://mlsanity.com/paper/2309.02155/) (2023)
- [Distance Matters in Human-Object Interaction Detection](https://mlsanity.com/paper/2207.01869/) (2022)
- [Improving Human-Object Interaction Detection via Virtual Image Learning](https://mlsanity.com/paper/2308.02606/) (2023)

## Built on by (most cited first, 6 of 6)

- [Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding](https://mlsanity.com/paper/2503.00361/) (2025)
- [Learning Human-Object Interaction as Groups](https://mlsanity.com/paper/2510.18357/) (2025)
- [SPU-IMR: Self-supervised Arbitrary-scale Point Cloud Upsampling via Iterative Mask-recovery Network](https://mlsanity.com/paper/2502.19452/) (2025)
- [DQEN: Dual Query Enhancement Network for DETR-based HOI Detection](https://mlsanity.com/paper/2508.18896/) (2025)
- [On the Robustness of Human-Object Interaction Detection against Distribution Shift](https://mlsanity.com/paper/2506.18021/) (2025)
- [Contextualized Representation Learning for Effective Human-Object Interaction Detection](https://mlsanity.com/paper/2509.12784/) (2025)

## Abstract (arXiv)

Human-object interactions (HOI) detection aims at capturing human-object pairs in images and corresponding actions. It is an important step toward high-level visual reasoning and scene understanding. However, due to the natural bias from the real world, existing methods mostly struggle with rare human-object pairs and lead to sub-optimal results. Recently, with the development of the generative model, a straightforward approach is to construct a more balanced dataset based on a group of supplementary samples. Unfortunately, there is a significant domain gap between the generated data and the original data, and simply merging the generated images into the original dataset cannot significantly boost the performance. To alleviate the above problem, we present a novel model-agnostic framework called Context-Enhanced Feature Alignment (CEFA) module, which can effectively align the generated data with the original data at the feature level and bridge the domain gap. Specifically, CEFA consists of a feature alignment module and a context enhancement module. On one hand, considering the crucial role of human-object pairs information in HOI tasks, the feature alignment module aligns the human-object pairs by aggregating instance information. On the other hand, to mitigate the issue of losing important context information caused by the traditional discriminator-style alignment method, we employ a context-enhanced image reconstruction module to improve the model's learning ability of contextual cues. Extensive experiments have shown that our method can serve as a plug-and-play module to improve the detection performance of HOI models on rare categoriesfootnotehttps://github.com/LijunZhang01/CEFA.