# Diagnosis of Human Object Interaction Detectors for Real World Educational Applications

arXiv: https://arxiv.org/abs/2606.02789 · HTML: https://mlsanity.com/paper/2606.02789/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [Getting to Know Low-light Images with The Exclusively Dark Dataset](https://mlsanity.com/paper/1805.11227/) (2018)
- [TIDE: A General Toolbox for Identifying Object Detection Errors](https://mlsanity.com/paper/2008.08115/) (2020)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [Learning Human-Object Interaction Detection using Interaction Points](https://mlsanity.com/paper/2003.14023/) (2020)
- [Mining the Benefits of Two-stage and One-stage HOI Detection](https://mlsanity.com/paper/2108.05077/) (2021)
- [HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Models](https://mlsanity.com/paper/2303.15786/) (2023)
- [Exploring Predicate Visual Context in Detecting Human-Object Interactions](https://mlsanity.com/paper/2308.06202/) (2023)
- [Diagnosing Human-object Interaction Detectors](https://mlsanity.com/paper/2308.08529/) (2023)
- [Trainee Action Recognition through Interaction Analysis in CCATT Mixed-Reality Training](https://mlsanity.com/paper/2509.17888/) (2025)

## Abstract (arXiv)

Human-object interaction (HOI) recognition is critical for automatically analyzing student behavior in complex educational environments. Although state-of-the-art (SOTA) HOI detectors perform well on benchmark datasets, their performance often degrades when deployed in real-world training environments due to domain-specific objects, occlusions, and complex visual conditions. In this paper, we introduce a diagnosis-driven framework that integrates a triplet-level HOI error taxonomy with error-factor attribution analysis for real-world educational video data. We study this problem in the context of Critical Care Air Transport Team (CCATT) mixed-reality medical training. Based on an analysis of HOI failure modes and their causes, we develop a diagnosis-informed refinement strategy for adapting pretrained HOI models to the target domain. Experiments on the CCATT dataset show that this approach improves the macro-F1 score of a pretrained CDN model from 48.6 to 90.2 through targeted refinement guided by diagnosed error factors. These results highlight the value of detailed diagnostic analysis for informing targeted adaptation of HOI models in real-world educational environments.