mlsanityData to 7 Oct 2026

Pi-HOC: Pairwise 3D Human-Object Contact Estimation

arXiv 2604.12923 ↗First posted 14 Apr 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Resolving real-world human-object interactions in images is a many-to-many challenge, in which disentangling fine-grained concurrent physical contact is particularly difficult. Existing semantic contact estimation methods are either limited to single-human settings or require object geometries (e.g., meshes) in addition to the input image. Current state-of-the-art leverages powerful VLM for category-level semantics but struggles with multi-human scenarios and scales poorly in inference. We introduce Pi-HOC, a single-pass, instance-aware framework for dense 3D semantic contact prediction of all human-object pairs. Pi-HOC detects instances, creates dedicated human-object (HO) tokens for each pair, and refines them using an InteractionFormer. A SAM-based decoder then predicts dense contact on SMPL human meshes for each human-object pair. On the MMHOI and DAMON datasets, Pi-HOC significantly improves accuracy and localization over state-of-the-art methods while achieving 20x higher throughput. We further demonstrate that predicted contacts improve SAM-3D image-to-mesh reconstruction via a test-time optimization algorithm and enable referential contact prediction from language queries without additional training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

31 of 38 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2017Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentationscited by 614
  7. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  8. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  9. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  10. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 176
  11. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  12. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
19 more
  1. 2022CHORE: Contact, Human and Object REconstruction from a single RGB imagecited by 95
  2. 2021Efficient Two-Stage Detection of Human-Object Interactions with a Novel Unary-Pairwise Transformercited by 91
  3. 2021On Self-Contact and Human Posecited by 88
  4. 2022Compositional Human-Scene Interaction Synthesis with Semantic Controlcited by 82
  5. 2026SAM 3D Body: Robust Full-Body Human Mesh Recoverycited by 61
  6. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  7. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  8. 2023Detecting Human-Object Contact in Imagescited by 41
  9. 2020Learning Complex 3D Human Self-Contactcited by 40
  10. 2023Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memorycited by 38
  11. 2024EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wildcited by 33
  12. 2022HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidancecited by 29
  13. 2024Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformercited by 29
  14. 2025Joint Optimization for 4D Human-Scene Reconstruction in the Wildcited by 29
  15. 2025PICO: Reconstructing 3D People In Contact with Objectscited by 22
  16. 2025InteractVLM: 3D Interaction Reasoning from 2D Foundational Modelscited by 21
  17. 2025PhySIC: Physically Plausible 3D Human-Scene Interaction and Contact from a Single Imagecited by 13
  18. 2025Locality-Aware Zero-Shot Human-Object Interaction Detectioncited by 12
  19. 2025MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactionscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DecoDINO: 3D Human-Scene Contact Prediction with Semantic Classification
  2. 2024Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models
  3. 2024Pose Priors from Language Models
  4. 2022Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors
  5. 2026Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models
  6. 2025InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing
  7. 2024ContactGen: Contact-Guided Interactive 3D Human Generation for Partners
  8. 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation