mlsanityData to 14 Oct 2026

RoHOI: Robustness Benchmark for Human-Object Interaction Detection

arXiv 2507.09111 ↗First posted 12 Jul 2025cs.CV cs.HC cs.ROcited by 3

Not on the map. Why?

Abstract (arXiv)

Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support. However, models trained on clean datasets degrade in real-world conditions due to unforeseen corruptions, leading to inaccurate predictions. To address this, we introduce the first robustness benchmark for HOI detection, evaluating model resilience under diverse challenges. Despite advances, current models struggle with environmental variability, occlusions, and noise. Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric. We systematically analyze existing models in the HOI field, revealing significant performance drops under corruptions. To improve robustness, we propose a Semantic-Aware Masking-based Progressive Learning (SAMPL) strategy to guide the model to be optimized based on holistic and partial cues, thus dynamically adjusting the model's optimization to enhance robust feature learning. Extensive experiments show that our approach outperforms state-of-the-art methods, setting a new standard for robust HOI detection. Benchmarks, datasets, and code are available at https://github.com/KratosWen/RoHOI.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 67 of its 87 references that are in the map. First posted on arXiv 12 Jul 2025.

Builds on

67 of 87 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2023Segment Anythingcited by 8,234
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  9. 2017Learning to Detect Human-Object Interactionscited by 240
  10. 2015Visual Semantic Role Labelingcited by 190
  11. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  12. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
38 more
  1. 2015Appearance-Based Gaze Estimation in the Wildcited by 153
  2. 2021QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Informationcited by 147
  3. 2019PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detectioncited by 143
  4. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 135
  5. 2022GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detectioncited by 120
  6. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  7. 2019Benchmarking the Robustness of Semantic Segmentation Modelscited by 111
  8. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  9. 2021Mining the Benefits of Two-stage and One-stage HOI Detectioncited by 103
  10. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 103
  11. 2024WAVES: Benchmarking the Robustness of Image Watermarkscited by 97
  12. 2020Spatially Conditioned Graphs for Detecting Human-Object Interactionscited by 93
  13. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  14. 2019Pose-aware Multi-level Feature Network for Human Object Interaction Detectioncited by 88
  15. 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
  16. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  17. 2021Detecting Human-Object Interaction via Fabricated Compositional Learningcited by 79
  18. 2021Affordance Transfer Learning for Human-Object Interaction Detectioncited by 79
  19. 2021Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detectioncited by 58
  20. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  21. 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 54
  22. 2022Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detectioncited by 47
  23. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 45
  24. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 41
  25. 2020Cascaded Human-Object Interaction Recognitioncited by 40
  26. 2018Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settingscited by 38
  27. 2023Relational Context Learning for Human-Object Interaction Detectioncited by 36
  28. 2018Pairwise Body-Part Attention for Recognizing Human-Object Interactionscited by 34
  29. 2022MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detectioncited by 34
  30. 2023Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Modelscited by 34
  31. 2023UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detectioncited by 31
  32. 2018Interact as You Intend: Intention-Driven Human-Object Interaction Detectioncited by 30
  33. 2023FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detectioncited by 30
  34. 2023ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editingcited by 30
  35. 2021QAHOI: Query-Based Anchors for Human-Object Interaction Detectioncited by 28
  36. 2024Is Your HD Map Constructor Reliable under Sensor Corruptions?cited by 28
  37. 2024EZ-HOI: VLM Adaptation via Guided Prompt Learning for Zero-Shot HOI Detectioncited by 28
  38. 2022TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibrationcited by 25

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Constructioncited by 2
  2. 2026Vision-Language Models for Ergonomic Assessment of Manual Lifting Tasks: Estimating Horizontal and Vertical Hand Distances from RGB Videocited by 0
  3. 2026HEIR: Learning Human-Entity Interactions with Functional Rolescited by 0