mlsanityData to 7 Oct 2026

Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detection

arXiv 2204.04836 ↗First posted 11 Apr 2022cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Human-Object Interaction detection is a holistic visual recognition task that entails object detection as well as interaction classification. Previous works of HOI detection has been addressed by the various compositions of subset predictions, e.g., Image -> HO -> I, Image -> HI -> O. Recently, transformer based architecture for HOI has emerged, which directly predicts the HOI triplets in an end-to-end fashion (Image -> HOI). Motivated by various inference paths for HOI detection, we propose cross-path consistency learning (CPC), which is a novel end-to-end learning strategy to improve HOI detection for transformers by leveraging augmented decoding paths. CPC learning enforces all the possible predictions from permuted inference sequences to be consistent. This simple scheme makes the model learn consistent representations, thereby improving generalization without increasing model capacity. Our experiments demonstrate the effectiveness of our method, and we achieved significant improvement on V-COCO and HICO-DET compared to the baseline models. Our code is available at https://github.com/mlvlab/CPChoi.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 51 of its 63 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

51 of 63 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  11. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  12. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
38 more
  1. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  2. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  4. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  5. 2017Deep Mutual Learningcited by 621
  6. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  7. 2021Conditional DETR for Fast Training Convergencecited by 422
  8. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  9. 2017Learning to Detect Human-Object Interactionscited by 238
  10. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
  11. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  12. 2015Visual Semantic Role Labelingcited by 188
  13. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  14. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  15. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  16. 2019Temporal Cycle-Consistency Learningcited by 151
  17. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  18. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  19. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 134
  20. 2020Visual Compositional Learning for Human-Object Interaction Detectioncited by 129
  21. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  22. 2020DRG: Dual Relation Graph for Human-Object Interaction Detectioncited by 117
  23. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  24. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  25. 2020Robust Learning Through Cross-Task Consistencycited by 105
  26. 2021Reformulating HOI Detection as Adaptive Set Predictioncited by 95
  27. 2020Detailed 2D-3D Joint Representation for Human-Object Interactioncited by 93
  28. 2016Learning Dense Correspondence via 3D-guided Cycle Consistencycited by 91
  29. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  30. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  31. 2020PaStaNet: Toward Human Activity Knowledge Enginecited by 79
  32. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  33. 2020Contextual Heterogeneous Graph Network for Human-Object Interaction Detectioncited by 50
  34. 2020ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detectioncited by 45
  35. 2021Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queriescited by 31
  36. 2020Polysemy Deciphering Network for Robust Human-Object Interaction Detectioncited by 25
  37. 2018Detecting Visual Relationships Using Box Attentioncited by 16
  38. 2017Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Taskscited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 4; 2025: 2; 2026: 2.

16 papers.

Sort:

By area:

  1. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 44
  2. 2022TokenMixup: Efficient Attention-guided Token-level Data Augmentation for Transformerscited by 26
  3. 2023Category Query Learning for Human-Object Interaction Classificationcited by 21
  4. 2024Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Modelscited by 15
  5. 2023Unified Visual Relationship Detection with Vision and Language Modelscited by 13
  6. 2023Human-Object Interaction Prediction in Videos through Gaze Followingcited by 13
  7. 2024Groupwise Query Specialization and Quality-Aware Multi-Assignment for Transformer-based Visual Relationship Detectioncited by 11
  8. 2023Improving Human-Object Interaction Detection via Virtual Image Learningcited by 4
  9. 2023DDS: Decoupled Dynamic Scene-Graph Generation Networkcited by 3
  10. 2025Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detectioncited by 3
  11. 2022TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistencycited by 1
  12. 2024SGTR+: End-to-end Scene Graph Generation with Transformercited by 1
4 more
  1. 2025Mitigating Long-Tail Bias in HOI Detection via Adaptive Diversity Cachecited by 1
  2. 2024STMixer: A One-Stage Sparse Action Detectorcited by 0
  3. 2026RegFormer: Transferable Relational Grounding for Efficient Weakly-Supervised Human-Object Interaction Detectioncited by 0
  4. 2026ScriptHOI: Learning Scripted State Transitions for Open-Vocabulary Human-Object Interaction Detectioncited by 0