mlsanityData to 7 Oct 2026

Human-Object Interaction Detection via Weak Supervision

arXiv 2112.00492 ↗First posted 1 Dec 2021cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

The goal of this paper is Human-object Interaction (HO-I) detection. HO-I detection aims to find interacting human-objects regions and classify their interaction from an image. Researchers obtain significant improvement in recent years by relying on strong HO-I alignment supervision from [5]. HO-I alignment supervision pairs humans with their interacted objects, and then aligns human-object pair(s) with their interaction categories. Since collecting such annotation is expensive, in this paper, we propose to detect HO-I without alignment supervision. We instead rely on image-level supervision that only enumerates existing interactions within the image without pointing where they happen. Our paper makes three contributions: i) We propose Align-Former, a visual-transformer based CNN that can detect HO-I with only image-level supervision. ii) Align-Former is equipped with HO-I align layer, that can learn to select appropriate targets to allow detector supervision. iii) We evaluate Align-Former on HICO-DET [5] and V-COCO [13], and show that Align-Former outperforms existing image-level supervised HO-I detectors by a large margin (4.71% mAP improvement from 16.14% to 20.85% on HICO-DET [5]).

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 1 Dec 2021.

Builds on

27 of 33 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2015Fast R-CNNcited by 2,930
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  9. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  10. 2017Learning to Detect Human-Object Interactionscited by 238
  11. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  12. 2015Visual Semantic Role Labelingcited by 188
15 more
  1. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  2. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  3. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  4. 2020Visual Compositional Learning for Human-Object Interaction Detectioncited by 129
  5. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  6. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  7. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  8. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  9. 2018You Look Twice: GaterNet for Dynamic Filter Selection in CNNscited by 46
  10. 2019Batch-Shaping for Learning Conditional Channel Gated Networkscited by 42
  11. 2017PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCNcited by 25
  12. 2020Detecting Human-Object Interaction with Mixed Supervisioncited by 9
  13. 2018No-Frills Human-Object Interaction Detection: Factorization, Layout Encodings, and Training Techniquescited by 8
  14. 2020Structured Visual Search via Composition-aware Learningcited by 5
  15. 2020Self-Selective Context for Interaction Recognitioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 0; 2026: 1.

5 papers.

  1. 2023Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learningcited by 11
  2. 2023Exploiting CLIP for Zero-shot HOI Detection Requires Knowledge Distillation at Multiple Levelscited by 5
  3. 2024A Review of Human-Object Interaction Detectioncited by 5
  4. 2026Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wildcited by 1
  5. 2024FreeA: Human-object Interaction Detection using Free Annotation Labelscited by 0