mlsanityData to 7 Oct 2026

Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning

arXiv 2303.01313 ↗First posted 2 Mar 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building-block for many vision tasks. One generalizable and scalable strategy for HOI detection is to use weak supervision, learning from image-level annotations only. This is inherently challenging due to ambiguous human-object associations, large search space of detecting HOIs and highly noisy training signal. A promising strategy to address those challenges is to exploit knowledge from large-scale pretrained models (e.g., CLIP), but a direct knowledge distillation strategy~citepliao2022gen does not perform well on the weakly-supervised setting. In contrast, we develop a CLIP-guided HOI representation capable of incorporating the prior knowledge at both image level and HOI instance level, and adopt a self-taught mechanism to prune incorrect human-object associations. Experimental results on HICO-DET and V-COCO show that our method outperforms the previous works by a sizable margin, showing the efficacy of our HOI representation.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

36 of 46 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Mask R-CNNcited by 7,259
  7. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  8. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  9. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  10. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  11. 2017Learning to Detect Human-Object Interactionscited by 238
  12. 2015Visual Semantic Role Labelingcited by 188
24 more
  1. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  2. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  3. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  4. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
  5. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  6. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  7. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  8. 2020Detailed 2D-3D Joint Representation for Human-Object Interactioncited by 93
  9. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  10. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  11. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  12. 2020Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detectioncited by 83
  13. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  14. 2020Cascaded Human-Object Interaction Recognitioncited by 40
  15. 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
  16. 2022What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactionscited by 35
  17. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  18. 2017PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCNcited by 25
  19. 2022Category-Aware Transformer Network for Better Human-Object Interaction Detectioncited by 22
  20. 2020Explanation-based Weakly-supervised Learning of Visual Relations with Graph Networkscited by 13
  21. 2018Transferable Interactiveness Knowledge for Human-Object Interaction Detectioncited by 12
  22. 2020Detecting Human-Object Interaction with Mixed Supervisioncited by 9
  23. 2021SDOF-Tracker: Fast and Accurate Multiple Human Tracking by Skipped-Detection and Optical-Flowcited by 7
  24. 2021Human-Object Interaction Detection via Weak Supervisioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 1; 2026: 2.

11 papers.

  1. 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 60
  2. 2024Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detectioncited by 7
  3. 2025Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibrationcited by 7
  4. 2023Exploiting CLIP for Zero-shot HOI Detection Requires Knowledge Distillation at Multiple Levelscited by 5
  5. 2024A Review of Human-Object Interaction Detectioncited by 5
  6. 2023Enhancing HOI Detection with Contextual Cues from Large Vision-Language Modelscited by 3
  7. 2024Towards Zero-shot Human-Object Interaction Detection via Vision-Language Integrationcited by 3
  8. 2026Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wildcited by 1
  9. 2024FreeA: Human-object Interaction Detection using Free Annotation Labelscited by 0
  10. 2024An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training setcited by 0
  11. 2026RegFormer: Transferable Relational Grounding for Efficient Weakly-Supervised Human-Object Interaction Detectioncited by 0