mlsanityData to 7 Oct 2026

Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detection

arXiv 2401.05676 ↗First posted 11 Jan 2024cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Human-Object Interaction (HOI) detection plays a vital role in scene understanding, which aims to predict the HOI triplet in the form of <human, object, action>. Existing methods mainly extract multi-modal features (e.g., appearance, object semantics, human pose) and then fuse them together to directly predict HOI triplets. However, most of these methods focus on seeking for self-triplet aggregation, but ignore the potential cross-triplet dependencies, resulting in ambiguity of action prediction. In this work, we propose to explore Self- and Cross-Triplet Correlations (SCTC) for HOI detection. Specifically, we regard each triplet proposal as a graph where Human, Object represent nodes and Action indicates edge, to aggregate self-triplet correlation. Also, we try to explore cross-triplet dependencies by jointly considering instance-level, semantic-level, and layout-level relations. Besides, we leverage the CLIP model to assist our SCTC obtain interaction-aware feature by knowledge distillation, which provides useful action clues for HOI detection. Extensive experiments on HICO-DET and V-COCO datasets verify the effectiveness of our proposed SCTC.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 39 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

32 of 39 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  9. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  10. 2017Learning to Detect Human-Object Interactionscited by 238
  11. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  12. 2015Visual Semantic Role Labelingcited by 188
20 more
  1. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  2. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  3. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  4. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  5. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  6. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
  7. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  8. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  9. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  10. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  11. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  12. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  13. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
  14. 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
  15. 2023UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detectioncited by 31
  16. 2022Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semanticscited by 22
  17. 2023Category Query Learning for Human-Object Interaction Classificationcited by 21
  18. 2021Improving Human-Object Interaction Detection via Phrase Learning and Label Compositioncited by 15
  19. 2023Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detectioncited by 13
  20. 2023Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 3.

7 papers.

  1. 2025Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibrationcited by 7
  2. 2025Learning Human-Object Interaction as Groupscited by 5
  3. 2024ContextHOI: Spatial Context Learning for Human-Object Interaction Detectioncited by 2
  4. 2025Mitigating Long-Tail Bias in HOI Detection via Adaptive Diversity Cachecited by 1
  5. 2026Mining Instance-Centric Vision-Language Contexts for Human-Object Interaction Detectioncited by 1
  6. 2026Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wildcited by 1
  7. 2026Denoise and Align: Diffusion-Driven Foreground Knowledge Prompting for Open-Vocabulary Temporal Action Detectioncited by 0