mlsanityData to 7 Oct 2026

Category Query Learning for Human-Object Interaction Classification

arXiv 2303.14005 ↗First posted 24 Mar 2023cs.CV cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

Unlike most previous HOI methods that focus on learning better human-object features, we propose a novel and complementary approach called category query learning. Such queries are explicitly associated to interaction categories, converted to image specific category representation via a transformer decoder, and learnt via an auxiliary image-level classification task. This idea is motivated by an earlier multi-label image classification method, but is for the first time applied for the challenging human-object interaction classification task. Our method is simple, general and effective. It is validated on three representative HOI baselines and achieves new state-of-the-art results on two benchmarks.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 43 of its 51 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

43 of 51 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  6. 2020Asymmetric Loss For Multi-Label Classificationcited by 379
  7. 2017Learning to Detect Human-Object Interactionscited by 238
  8. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  9. 2015Visual Semantic Role Labelingcited by 188
  10. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  11. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  12. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
31 more
  1. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  2. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  3. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  4. 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 126
  5. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
  6. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  7. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  8. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  9. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  10. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  11. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  12. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  13. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  14. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  15. 2020PaStaNet: Toward Human Activity Knowledge Enginecited by 79
  16. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  17. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  18. 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
  19. 2019Detecting Human-Object Interactions via Functional Generalizationcited by 53
  20. 2020Contextual Heterogeneous Graph Network for Human-Object Interaction Detectioncited by 50
  21. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 46
  22. 2022Interactiveness Field in Human-Object Interactionscited by 40
  23. 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
  24. 2022What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactionscited by 35
  25. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  26. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
  27. 2020Polysemy Deciphering Network for Robust Human-Object Interaction Detectioncited by 25
  28. 2022Category-Aware Transformer Network for Better Human-Object Interaction Detectioncited by 22
  29. 2022Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detectioncited by 16
  30. 2022Towards Hard-Positive Query Mining for DETR-based Human-Object Interaction Detectioncited by 12
  31. 2022RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentationcited by 11

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 8; 2025: 10.

21 papers.

Sort:

By area:

  1. 2023Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Modelscited by 34
  2. 2024EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detectioncited by 28
  3. 2024Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detectioncited by 26
  4. 2024Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detectioncited by 25
  5. 2024Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Modelscited by 15
  6. 2025HOLa: Zero-Shot HOI Detection with Low-Rank Decomposed VLM Feature Adaptationcited by 12
  7. 2024Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detectioncited by 7
  8. 2024Toward Multi-class Anomaly Detection: Exploring Class-aware Unified Model against Inter-class Interferencecited by 7
  9. 2025Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibrationcited by 7
  10. 2025Learning Human-Object Interaction as Groupscited by 5
  11. 2023Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisorcited by 4
  12. 2025VHOI: Controllable Video Generation of Human-Object Interactions from Sparse Trajectories via Motion Densificationcited by 4
9 more
  1. 2023Enhancing HOI Detection with Contextual Cues from Large Vision-Language Modelscited by 3
  2. 2024Geometric Features Enhanced Human-Object Interaction Detectioncited by 2
  3. 2024Orchestrating the Symphony of Prompt Distribution Learning for Human-Object Interaction Detectioncited by 2
  4. 2025CQ-DINO: Mitigating Gradient Dilution via Category Queries for Vast Vocabulary Object Detectioncited by 1
  5. 2025DQEN: Dual Query Enhancement Network for DETR-based HOI Detectioncited by 1
  6. 2025B-RIGHT: Benchmark Re-evaluation for Integrity in Generalized Human-Object Interaction Testingcited by 0
  7. 2025On the Robustness of Human-Object Interaction Detection against Distribution Shiftcited by 0
  8. 2025Contextualized Representation Learning for Effective Human-Object Interaction Detectioncited by 0
  9. 2025Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMscited by 0