Detecting Human-Object Interactions with Action Co-occurrence Priors
Not on the map. Why?
Abstract (arXiv)
A common problem in human-object interaction (HOI) detection task is that numerous HOI classes have only a small number of labeled examples, resulting in training sets with a long-tailed distribution. The lack of positive labels can lead to low classification accuracy for these classes. Towards addressing this issue, we observe that there exist natural correlations and anti-correlations among human-object interactions. In this paper, we model the correlations as action co-occurrence matrices and present techniques to learn these priors and leverage them for more effective training, especially in rare classes. The utility of our approach is demonstrated experimentally, where the performance of our approach exceeds the state-of-the-art methods on both of the two leading HOI detection benchmark datasets, HICO-Det and V-COCO.
Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jul 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 20; 2022: 23; 2023: 18; 2024: 7; 2025: 3; 2026: 1.
72 papers.
Sort:
By area:
- 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
- 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
- 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
- 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
- 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
- 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
- 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
- 2022A Survey on Long-Tailed Visual Recognitioncited by 75
- 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 60
- 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
- 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
- 2021DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learningcited by 51
38 more
- 2022Interactiveness Field in Human-Object Interactionscited by 40
- 2022Human-Object Interaction Detection via Disentangled Transformercited by 40
- 2023Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memorycited by 38
- 2022What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactionscited by 35
- 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
- 2021LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentationcited by 33
- 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
- 2022End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillationcited by 30
- 2022HAKE: A Knowledge Engine Foundation for Human Activity Understandingcited by 28
- 2023Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Modelscited by 27
- 2023Neural-Logic Human-Object Interaction Detectioncited by 25
- 2022Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semanticscited by 22
- 2023Category Query Learning for Human-Object Interaction Classificationcited by 21
- 2022Iwin: Human-Object Interaction Detection via Transformer with Irregular Windowscited by 19
- 2021MCDAL: Maximum Classifier Discrepancy for Active Learningcited by 18
- 2022Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detectioncited by 16
- 2021Improving Human-Object Interaction Detection via Phrase Learning and Label Compositioncited by 15
- 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
- 2021Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillationcited by 14
- 2022Video-based Human-Object Interaction Detection from Tubelet Tokenscited by 13
- 2023Unified Visual Relationship Detection with Vision and Language Modelscited by 13
- 2023Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detectioncited by 13
- 2022Towards Hard-Positive Query Mining for DETR-based Human-Object Interaction Detectioncited by 12
- 2022Generative Bias for Robust Visual Question Answeringcited by 12
- 2021ACP++: Action Co-occurrence Priors for Human-Object Interaction Detectioncited by 11
- 2022Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detectioncited by 9
- 2022Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detectioncited by 9
- 2023HODN: Disentangling Human-Object Feature for HOI Detectioncited by 9
- 2021GTNet:Guided Transformer Network for Detecting Human-Object Interactionscited by 8
- 2022LocATe: End-to-end Localization of Actions in 3D with Transformerscited by 8
- 2022Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognitioncited by 8
- 2023Agglomerative Transformer for Human-Object Interaction Detectioncited by 8
- 2023Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learningcited by 8
- 2021The Overlooked Classifier in Human-Object Interaction Recognitioncited by 7
- 2024Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detectioncited by 7
- 2024Seeing Beyond Classes: Zero-Shot Grounded Situation Recognition via Language Explainercited by 7
- 2021Single-Modal Entropy based Active Learning for Visual Question Answeringcited by 6
- 2023Weakly-Supervised HOI Detection from Interaction Labels Only and Language/Vision-Language Priorscited by 6
50 most cited of 72 All 72 (JSON)
- 2026Towards Unconstrained Human-Object Interaction15 Apr 2026
- 2025Learning Human-Object Interaction as Groups21 Oct 2025
- 2025No More Sibling Rivalry: Debiasing Human-Object Interaction Detection31 Aug 2025
- 2025QueryCraft: Transformer-Guided Query Initialization for Enhanced Human-Object Interaction Detection12 Aug 2025
- 2024A Review of Human-Object Interaction Detection20 Aug 2024
- 2024Towards Flexible Visual Relationship Segmentation15 Aug 2024
- 2024UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection14 Aug 2024
- 2024An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set11 Aug 2024
- 2024Seeing Beyond Classes: Zero-Shot Grounded Situation Recognition via Language Explainer24 Apr 2024
- 2024Enhancing Human-Centered Dynamic Scene Understanding via Multiple LLMs Collaborated Reasoning15 Mar 2024
- 2024Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detection11 Jan 2024
- 2023Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning26 Dec 2023
38 more
- 2023Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection4 Dec 2023
- 2023Neural-Logic Human-Object Interaction Detection16 Nov 2023
- 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection3 Nov 2023
- 2023Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Prediction12 Sep 2023
- 2023Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memory7 Sep 2023
- 2023HODN: Disentangling Human-Object Feature for HOI Detection20 Aug 2023
- 2023Agglomerative Transformer for Human-Object Interaction Detection16 Aug 2023
- 2023Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection25 Jul 2023
- 2023Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisor5 Jul 2023
- 2023HOKEM: Human and Object Keypoint-based Extension Module for Human-Object Interaction Detection25 Jun 2023
- 2023Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Models21 May 2023
- 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Models28 Mar 2023
- 2023Category Query Learning for Human-Object Interaction Classification24 Mar 2023
- 2023Unified Visual Relationship Detection with Vision and Language Models16 Mar 2023
- 2023Weakly-Supervised HOI Detection from Interaction Labels Only and Language/Vision-Language Priors9 Mar 2023
- 2023DDS: Decoupled Dynamic Scene-Graph Generation Network18 Jan 2023
- 2023Parallel Reasoning Network for Human-Object Interaction Detection9 Jan 2023
- 2022Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition1 Nov 2022
- 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection5 Sep 2022
- 2022Generative Bias for Robust Visual Question Answering1 Aug 2022
- 2022Knowledge Guided Bidirectional Attention Network for Human-Object Interaction Detection16 Jul 2022
- 2022Towards Hard-Positive Query Mining for DETR-based Human-Object Interaction Detection12 Jul 2022
- 2022A Skeleton-aware Graph Convolutional Network for Human-Object Interaction Detection11 Jul 2022
- 2022Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detection6 Jul 2022
- 2022Video-based Human-Object Interaction Detection from Tubelet Tokens4 Jun 2022
- 2022A Survey on Long-Tailed Visual Recognition27 May 2022
- 2022Human-Object Interaction Detection via Disentangled Transformer20 Apr 2022
- 2022Interactiveness Field in Human-Object Interactions16 Apr 2022
- 2022Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detection11 Apr 2022
- 2022What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactions2 Apr 2022
- 2022End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation1 Apr 2022
- 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detection28 Mar 2022
- 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection26 Mar 2022
- 2022LocATe: End-to-end Localization of Actions in 3D with Transformers21 Mar 2022
- 2022Iwin: Human-Object Interaction Detection via Transformer with Irregular Windows20 Mar 2022
- 2022The Overlooked Classifier in Human-Object Interaction Recognition10 Mar 2022
- 2022Effective Actor-centric Human-object Interaction Detection24 Feb 2022
- 2022Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detection19 Feb 2022
50 newest of 72 All 72 (JSON)
- 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 60
- 2023Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Modelscited by 27
- 2024Seeing Beyond Classes: Zero-Shot Grounded Situation Recognition via Language Explainercited by 7
- 2023Weakly-Supervised HOI Detection from Interaction Labels Only and Language/Vision-Language Priorscited by 6
- 2023Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisorcited by 4
- 2024Seeing Beyond Classes: Zero-Shot Grounded Situation Recognition via Language Explainer24 Apr 2024
- 2023Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisor5 Jul 2023
- 2023Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Models21 May 2023
- 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Models28 Mar 2023
- 2023Weakly-Supervised HOI Detection from Interaction Labels Only and Language/Vision-Language Priors9 Mar 2023
- 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
- 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
- 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
- 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 119
- 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
- 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
- 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
- 2022A Survey on Long-Tailed Visual Recognitioncited by 75
- 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
- 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
- 2021DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learningcited by 51
- 2022Interactiveness Field in Human-Object Interactionscited by 40
12 most cited of 66 All 66 (JSON)
- 2026Towards Unconstrained Human-Object Interaction15 Apr 2026
- 2025Learning Human-Object Interaction as Groups21 Oct 2025
- 2025No More Sibling Rivalry: Debiasing Human-Object Interaction Detection31 Aug 2025
- 2025QueryCraft: Transformer-Guided Query Initialization for Enhanced Human-Object Interaction Detection12 Aug 2025
- 2024A Review of Human-Object Interaction Detection20 Aug 2024
- 2024Towards Flexible Visual Relationship Segmentation15 Aug 2024
- 2024UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection14 Aug 2024
- 2024An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set11 Aug 2024
- 2024Exploring Self- and Cross-Triplet Correlations for Human-Object Interaction Detection11 Jan 2024
- 2023Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning26 Dec 2023
- 2023Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection4 Dec 2023
- 2023Neural-Logic Human-Object Interaction Detection16 Nov 2023
12 newest of 66 All 66 (JSON)