# FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detection

arXiv: https://arxiv.org/abs/2301.04019 · HTML: https://mlsanity.com/paper/2301.04019/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 65 references that are in the map.
- First posted on arXiv 8 Jan 2023.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [Detecting and Recognizing Human-Object Interactions](https://mlsanity.com/paper/1704.07333/) (2017)
- [Visual Semantic Role Labeling](https://mlsanity.com/paper/1505.04474/) (2015)
- [Dynamic Head: Unifying Object Detection Heads with Attentions](https://mlsanity.com/paper/2106.08322/) (2021)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [Learning Human-Object Interactions by Graph Parsing Neural Networks](https://mlsanity.com/paper/1808.07962/) (2018)
- [QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information](https://mlsanity.com/paper/2103.05399/) (2021)
- [Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2105.15075/) (2021)
- [PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection](https://mlsanity.com/paper/1912.12898/) (2019)
- [End-to-End Human Object Interaction Detection with HOI Transformer](https://mlsanity.com/paper/2103.04503/) (2021)
- [Visual Compositional Learning for Human-Object Interaction Detection](https://mlsanity.com/paper/2007.12407/) (2020)

## Built on by (most cited first, 30 of 30)

- [Open-World Human-Object Interaction Detection via Multi-modal Prompts](https://mlsanity.com/paper/2406.07221/) (2024)
- [CoReS: Orchestrating the Dance of Reasoning and Segmentation](https://mlsanity.com/paper/2404.05673/) (2024)
- [Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Model](https://mlsanity.com/paper/2305.12252/) (2023)
- [Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captions](https://mlsanity.com/paper/2311.17048/) (2023)
- [Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenario](https://mlsanity.com/paper/2306.12152/) (2023)
- [InteractDiffusion: Interaction Control in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2312.05849/) (2023)
- [Diagnosing Human-object Interaction Detectors](https://mlsanity.com/paper/2308.08529/) (2023)
- [CycleHOI: Improving Human-Object Interaction Detection with Cycle Consistency of Detection and Generation](https://mlsanity.com/paper/2407.11433/) (2024)
- [A Review of Human-Object Interaction Detection](https://mlsanity.com/paper/2408.10641/) (2024)
- [An Image-like Diffusion Method for Human-Object Interaction Detection](https://mlsanity.com/paper/2503.18134/) (2025)
- [Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisor](https://mlsanity.com/paper/2307.02291/) (2023)
- [Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models](https://mlsanity.com/paper/2311.16475/) (2023)
- [Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videos](https://mlsanity.com/paper/2506.03440/) (2025)
- [RoHOI: Robustness Benchmark for Human-Object Interaction Detection](https://mlsanity.com/paper/2507.09111/) (2025)
- [No More Sibling Rivalry: Debiasing Human-Object Interaction Detection](https://mlsanity.com/paper/2509.00760/) (2025)
- [Towards Flexible Visual Relationship Segmentation](https://mlsanity.com/paper/2408.08305/) (2024)
- [ContextHOI: Spatial Context Learning for Human-Object Interaction Detection](https://mlsanity.com/paper/2412.09050/) (2024)
- [Streamlined Open-Vocabulary Human-Object Interaction Detection](https://mlsanity.com/paper/2603.27500/) (2026)
- [TMHOI: Translational Model for Human-Object Interaction Detection](https://mlsanity.com/paper/2303.04253/) (2023)
- [HICO-DET-SG and V-COCO-SG: New Data Splits for Evaluating the Systematic Generalization Performance of Human-Object Interaction Detection Models](https://mlsanity.com/paper/2305.09948/) (2023)
- [VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysis](https://mlsanity.com/paper/2411.18038/) (2024)
- [Mitigating Long-Tail Bias in HOI Detection via Adaptive Diversity Cache](https://mlsanity.com/paper/2511.18811/) (2025)
- [HOIBlender: Blending Lightweight Detection with Vision-Language Priors for Efficient Human-Object Interaction Detection](https://mlsanity.com/paper/2609.23431/) (2026)
- [Detection and Localization of Firearm Carriers in Complex Scenes for Improved Safety Measures](https://mlsanity.com/paper/2309.09236/) (2023)
- [Efficient Human-Object-Interaction (EHOI) Detection via Interaction Label Coding and Conditional Decision](https://mlsanity.com/paper/2408.07018/) (2024)
- [Improving Fine-Grained Control via Aggregation of Multiple Diffusion Models](https://mlsanity.com/paper/2410.01262/) (2024)
- [Conceptualizing Multi-scale Wavelet Attention and Ray-based Encoding for Human-Object Interaction Detection](https://mlsanity.com/paper/2507.10977/) (2025)
- [Funnel-HOI: Top-Down Perception for Zero-Shot HOI Detection](https://mlsanity.com/paper/2507.12628/) (2025)
- [Multimodal Graph Network Modeling for Human-Object Interaction Detection with PDE Graph Diffusion](https://mlsanity.com/paper/2509.12554/) (2025)
- [Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMs](https://mlsanity.com/paper/2512.17640/) (2025)

## Abstract (arXiv)

Human-Object Interaction (HOI), as an important problem in computer vision, requires locating the human-object pair and identifying the interactive relationships between them. The HOI instance has a greater span in spatial, scale, and task than the individual object instance, making its detection more susceptible to noisy backgrounds. To alleviate the disturbance of noisy backgrounds on HOI detection, it is necessary to consider the input image information to generate fine-grained anchors which are then leveraged to guide the detection of HOI instances. However, it is challenging for the following reasons. i) how to extract pivotal features from the images with complex background information is still an open question. ii) how to semantically align the extracted features and query embeddings is also a difficult issue. In this paper, a novel end-to-end transformer-based framework (FGAHOI) is proposed to alleviate the above problems. FGAHOI comprises three dedicated components namely, multi-scale sampling (MSS), hierarchical spatial-aware merging (HSAM) and task-aware merging mechanism (TAM). MSS extracts features of humans, objects and interaction areas from noisy backgrounds for HOI instances of various scales. HSAM and TAM semantically align and merge the extracted features and query embeddings in the hierarchical spatial and task perspectives in turn. In the meanwhile, a novel training strategy Stage-wise Training Strategy is designed to reduce the training pressure caused by overly complex tasks done by FGAHOI. In addition, we propose two ways to measure the difficulty of HOI detection and a novel dataset, i.e., HOI-SDC for the two challenges (Uneven Distributed Area in Human-Object Pairs and Long Distance Visual Modeling of Human-Object Pairs) of HOI instances detection.