# Prompt Guidance and Human Proximal Perception for HOT Prediction with Regional Joint Loss

arXiv: https://arxiv.org/abs/2507.01630 · HTML: https://mlsanity.com/paper/2507.01630/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [Detecting and Recognizing Human-Object Interactions](https://mlsanity.com/paper/1704.07333/) (2017)
- [Visual Semantic Role Labeling](https://mlsanity.com/paper/1505.04474/) (2015)
- [End-to-End Human Object Interaction Detection with HOI Transformer](https://mlsanity.com/paper/2103.04503/) (2021)
- [GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection](https://mlsanity.com/paper/2203.13954/) (2022)
- [PaStaNet: Toward Human Activity Knowledge Engine](https://mlsanity.com/paper/2004.00945/) (2020)
- [DECO: Dense Estimation of 3D Human-Scene Contact In The Wild](https://mlsanity.com/paper/2309.15273/) (2023)
- [Detecting Human-Object Contact in Images](https://mlsanity.com/paper/2303.03373/) (2023)
- [Detecting Hands and Recognizing Physical Contact in the Wild](https://mlsanity.com/paper/2010.09676/) (2020)
- [A Dataset of Relighted 3D Interacting Hands](https://mlsanity.com/paper/2310.17768/) (2023)
- [Open-World Human-Object Interaction Detection via Multi-modal Prompts](https://mlsanity.com/paper/2406.07221/) (2024)
- [CycleHOI: Improving Human-Object Interaction Detection with Cycle Consistency of Detection and Generation](https://mlsanity.com/paper/2407.11433/) (2024)
- [A Review of Human-Object Interaction Detection](https://mlsanity.com/paper/2408.10641/) (2024)
- [Precision-Enhanced Human-Object Contact Detection via Depth-Aware Perspective Interaction and Object Texture Restoration](https://mlsanity.com/paper/2412.09920/) (2024)
- [Estimation of posture and joint angle of human body using foot pressure distribution: Morphological computation with human foot](https://mlsanity.com/paper/2401.12464/) (2024)

## Built on by (most cited first, 1 of 1)

- [What-Meets-Where: Unified Learning of Action and Contact Localization in Images](https://mlsanity.com/paper/2508.09428/) (2025)

## Abstract (arXiv)

The task of Human-Object conTact (HOT) detection involves identifying the specific areas of the human body that are touching objects. Nevertheless, current models are restricted to just one type of image, often leading to too much segmentation in areas with little interaction, and struggling to maintain category consistency within specific regions. To tackle this issue, a HOT framework, termed P3HOT, is proposed, which blends Prompt guidance and human Proximal Perception. To begin with, we utilize a semantic-driven prompt mechanism to direct the network's attention towards the relevant regions based on the correlation between image and text. Then a human proximal perception mechanism is employed to dynamically perceive key depth range around the human, using learnable parameters to effectively eliminate regions where interactions are not expected. Calculating depth resolves the uncertainty of the overlap between humans and objects in a 2D perspective, providing a quasi-3D viewpoint. Moreover, a Regional Joint Loss (RJLoss) has been created as a new loss to inhibit abnormal categories in the same area. A new evaluation metric called ``AD-Acc.'' is introduced to address the shortcomings of existing methods in addressing negative samples. Comprehensive experimental results demonstrate that our approach achieves state-of-the-art performance in four metrics across two benchmark datasets. Specifically, our model achieves an improvement of 0.7uparrow, 2.0uparrow, 1.6uparrow, and 11.0uparrow in SC-Acc., mIoU, wIoU, and AD-Acc. metrics, respectively, on the HOT-Annotated dataset. The sources code are available at https://github.com/YuxiaoWang-AI/P3HOT.