mlsanityData to 7 Oct 2026

Iwin: Human-Object Interaction Detection via Transformer with Irregular Windows

arXiv 2203.10537 ↗First posted 20 Mar 2022cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper presents a new vision Transformer, named Iwin Transformer, which is specifically designed for human-object interaction (HOI) detection, a detailed scene understanding task involving a sequential process of human/object detection and interaction recognition. Iwin Transformer is a hierarchical Transformer which progressively performs token representation learning and token agglomeration within irregular windows. The irregular windows, achieved by augmenting regular grid locations with learned offsets, 1) eliminate redundancy in token representation learning, which leads to efficient human/object detection, and 2) enable the agglomerated tokens to align with humans/objects with different shapes, which facilitates the acquisition of highly-abstracted visual semantics for interaction recognition. The effectiveness and efficiency of Iwin Transformer are verified on the two standard HOI detection benchmark datasets, HICO-DET and V-COCO. Results show our method outperforms existing Transformers-based methods by large margins (3.7 mAP gain on HICO-DET and 2.0 mAP gain on V-COCO) with fewer training epochs (0.5 ×).

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 20 Mar 2022.

Builds on

40 of 53 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2017Mask R-CNNcited by 7,259
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  11. 2017Deformable Convolutional Networkscited by 1,481
  12. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
28 more
  1. 2017Learning to Detect Human-Object Interactionscited by 238
  2. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  3. 2015Visual Semantic Role Labelingcited by 188
  4. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  5. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  6. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  7. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  8. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  9. 2020Visual Compositional Learning for Human-Object Interaction Detectioncited by 129
  10. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  11. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  12. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  13. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  14. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  15. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  16. 2020Detailed 2D-3D Joint Representation for Human-Object Interactioncited by 93
  17. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  18. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  19. 2020PaStaNet: Toward Human Activity Knowledge Enginecited by 79
  20. 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
  21. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  22. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  23. 2019Deep Contextual Attention for Human-Object Interaction Detectioncited by 54
  24. 2020Contextual Heterogeneous Graph Network for Human-Object Interaction Detectioncited by 50
  25. 2020Cascaded Human-Object Interaction Recognitioncited by 40
  26. 2018Interact as You Intend: Intention-Driven Human-Object Interaction Detectioncited by 30
  27. 2020Polysemy Deciphering Network for Robust Human-Object Interaction Detectioncited by 25
  28. 2020A Graph-based Interactive Reasoning for Human-Object Interaction Detectioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 6; 2025: 2; 2026: 2.

19 papers.

Sort:

By area:

  1. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  2. 2023Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memorycited by 38
  3. 2023Neural-Logic Human-Object Interaction Detectioncited by 25
  4. 2024Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Modelscited by 15
  5. 2024Nonverbal Interaction Detectioncited by 14
  6. 2022Video-based Human-Object Interaction Detection from Tubelet Tokenscited by 13
  7. 2025Locality-Aware Zero-Shot Human-Object Interaction Detectioncited by 12
  8. 2023Agglomerative Transformer for Human-Object Interaction Detectioncited by 8
  9. 2023Joint Gaze-Location and Gaze-Object Detectioncited by 6
  10. 2024A Review of Human-Object Interaction Detectioncited by 5
  11. 2023ECEA: Extensible Co-Existing Attention for Few-Shot Object Detectioncited by 4
  12. 2023HiH: A Multi-modal Hierarchy in Hierarchy Network for Unconstrained Gait Recognitioncited by 4
7 more
  1. 2024Towards Zero-shot Human-Object Interaction Detection via Vision-Language Integrationcited by 3
  2. 2024MVBIND: Self-Supervised Music Recommendation For Videos Via Embedding Space Bindingcited by 2
  3. 2024Quality-guided Skin Tone Enhancement for Portrait Photographycited by 2
  4. 2025Dynamic Scoring with Enhanced Semantics for Training-Free Human-Object Interaction Detectioncited by 2
  5. 2023Sharingan: A Transformer-based Architecture for Gaze Followingcited by 1
  6. 2026Zero-shot HOI Detection with MLLM-based Detector-agnostic Interaction Recognitioncited by 1
  7. 2026FineHOI: Part-Aware Dense Representations for Zero-Shot Human-Object Interaction Detectioncited by 0