mlsanityData to 7 Oct 2026

FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detection

arXiv 2301.04019 ↗First posted 8 Jan 2023cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

Human-Object Interaction (HOI), as an important problem in computer vision, requires locating the human-object pair and identifying the interactive relationships between them. The HOI instance has a greater span in spatial, scale, and task than the individual object instance, making its detection more susceptible to noisy backgrounds. To alleviate the disturbance of noisy backgrounds on HOI detection, it is necessary to consider the input image information to generate fine-grained anchors which are then leveraged to guide the detection of HOI instances. However, it is challenging for the following reasons. i) how to extract pivotal features from the images with complex background information is still an open question. ii) how to semantically align the extracted features and query embeddings is also a difficult issue. In this paper, a novel end-to-end transformer-based framework (FGAHOI) is proposed to alleviate the above problems. FGAHOI comprises three dedicated components namely, multi-scale sampling (MSS), hierarchical spatial-aware merging (HSAM) and task-aware merging mechanism (TAM). MSS extracts features of humans, objects and interaction areas from noisy backgrounds for HOI instances of various scales. HSAM and TAM semantically align and merge the extracted features and query embeddings in the hierarchical spatial and task perspectives in turn. In the meanwhile, a novel training strategy Stage-wise Training Strategy is designed to reduce the training pressure caused by overly complex tasks done by FGAHOI. In addition, we propose two ways to measure the difficulty of HOI detection and a novel dataset, i.e., HOI-SDC for the two challenges (Uneven Distributed Area in Human-Object Pairs and Long Distance Visual Modeling of Human-Object Pairs) of HOI instances detection.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 49 of its 65 references that are in the map. First posted on arXiv 8 Jan 2023.

Builds on

49 of 65 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  10. 2015Fast R-CNNcited by 2,930
  11. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  12. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
37 more
  1. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  2. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  3. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  4. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  5. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  6. 2022Vision Transformer with Deformable Attentioncited by 293
  7. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  8. 2017Learning to Detect Human-Object Interactionscited by 238
  9. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  10. 2015Visual Semantic Role Labelingcited by 188
  11. 2021Dynamic Head: Unifying Object Detection Heads with Attentionscited by 185
  12. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  13. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  14. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  15. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  16. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  17. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  18. 2020Visual Compositional Learning for Human-Object Interaction Detectioncited by 129
  19. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  20. 2021RegionViT: Regional-to-Local Attention for Vision Transformerscited by 126
  21. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  22. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  23. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  24. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  25. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  26. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  27. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  28. 2018Pose Flow: Efficient Online Pose Trackingcited by 80
  29. 2020PaStaNet: Toward Human Activity Knowledge Enginecited by 79
  30. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  31. 2022Accelerating DETR Convergence via Semantic-Aligned Matchingcited by 58
  32. 2019Detecting Human-Object Interactions via Functional Generalizationcited by 53
  33. 2017FSSD: Feature Fusion Single Shot Multibox Detectorcited by 42
  34. 2021QAHOI: Query-Based Anchors for Human-Object Interaction Detectioncited by 28
  35. 2018Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfercited by 27
  36. 2020Polysemy Deciphering Network for Robust Human-Object Interaction Detectioncited by 25
  37. 2022Conditional DETR V2: Efficient Detection Transformer with Box Queriescited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 9; 2025: 9; 2026: 2.

30 papers.

Sort:

By area:

  1. 2024Open-World Human-Object Interaction Detection via Multi-modal Promptscited by 28
  2. 2024CoReS: Orchestrating the Dance of Reasoning and Segmentationcited by 25
  3. 2023Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Modelcited by 21
  4. 2023Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captionscited by 21
  5. 2023Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenariocited by 14
  6. 2023InteractDiffusion: Interaction Control in Text-to-Image Diffusion Modelscited by 13
  7. 2023Diagnosing Human-object Interaction Detectorscited by 11
  8. 2024CycleHOI: Improving Human-Object Interaction Detection with Cycle Consistency of Detection and Generationcited by 5
  9. 2024A Review of Human-Object Interaction Detectioncited by 5
  10. 2025An Image-like Diffusion Method for Human-Object Interaction Detectioncited by 5
  11. 2023Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisorcited by 4
  12. 2023Enhancing HOI Detection with Contextual Cues from Large Vision-Language Modelscited by 3
18 more
  1. 2025Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videoscited by 3
  2. 2025RoHOI: Robustness Benchmark for Human-Object Interaction Detectioncited by 3
  3. 2025No More Sibling Rivalry: Debiasing Human-Object Interaction Detectioncited by 3
  4. 2024Towards Flexible Visual Relationship Segmentationcited by 2
  5. 2024ContextHOI: Spatial Context Learning for Human-Object Interaction Detectioncited by 2
  6. 2026Streamlined Open-Vocabulary Human-Object Interaction Detectioncited by 2
  7. 2023TMHOI: Translational Model for Human-Object Interaction Detectioncited by 1
  8. 2023HICO-DET-SG and V-COCO-SG: New Data Splits for Evaluating the Systematic Generalization Performance of Human-Object Interaction Detection Modelscited by 1
  9. 2024VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysiscited by 1
  10. 2025Mitigating Long-Tail Bias in HOI Detection via Adaptive Diversity Cachecited by 1
  11. 2023Detection and Localization of Firearm Carriers in Complex Scenes for Improved Safety Measurescited by 0
  12. 2024Efficient Human-Object-Interaction (EHOI) Detection via Interaction Label Coding and Conditional Decisioncited by 0
  13. 2024Improving Fine-Grained Control via Aggregation of Multiple Diffusion Modelscited by 0
  14. 2025Conceptualizing Multi-scale Wavelet Attention and Ray-based Encoding for Human-Object Interaction Detectioncited by 0
  15. 2025Funnel-HOI: Top-Down Perception for Zero-Shot HOI Detectioncited by 0
  16. 2025Multimodal Graph Network Modeling for Human-Object Interaction Detection with PDE Graph Diffusioncited by 0
  17. 2025Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMscited by 0
  18. 2026HOIBlender: Blending Lightweight Detection with Vision-Language Priors for Efficient Human-Object Interaction Detectioncited by 0