mlsanityData to 14 Oct 2026

Improving and Evaluating Hand-Object Interaction Detection

arXiv 2606.17384 ↗First posted 16 Jun 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Understanding hands and the objects they interact with, both directly and through tools, is a key step for tasks ranging from action perception to 3D reconstruction and robotics. Our paper provides several contributions to the Hand-Object Interaction (HOI) understanding literature: (1) HOI-DETR, a new framework that introduces hand-object and object-object interactions to the Co-DETR architecture to produce a state-of-the-art method; (2) a comprehensive HOI evaluation suite of 4 diverse datasets, including a video benchmark derived from the HD-EPIC dataset and fresh annotations that improve the Hands23 benchmark and (3) a trained checkpoint that significantly improves the state of the art across Hands23, HOIST, FineBio, and HD-EPIC, including mAP gains of over 20 percentage points on Hands23 and FineBio. Our ablations confirm the contributions of each model component.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 53 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

42 of 53 references.

References · 42

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2023Segment Anythingcited by 8,234
  5. 2017Mask R-CNNcited by 7,277
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  8. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  9. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  10. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  11. 2020Rescaling Egocentric Visioncited by 540
  12. 2023Segment Anything in High Qualitycited by 376
30 more
  1. 2023Reconstructing Hands in 3D with Transformerscited by 306
  2. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  3. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  4. 2017Learning to Detect Human-Object Interactionscited by 240
  5. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  6. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  7. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  8. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 183
  9. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  10. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 161
  11. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  12. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  13. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 140
  14. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 127
  15. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 124
  16. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 104
  17. 2022Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applicationscited by 78
  18. 2017Action Tubelet Detector for Spatio-Temporal Action Localizationcited by 77
  19. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  20. 2020Actions as Moving Pointscited by 55
  21. 2020Detecting Hands and Recognizing Physical Contact in the Wildcited by 38
  22. 2022Understanding 3D Object Articulation in Internet Videoscited by 36
  23. 2024Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mindcited by 28
  24. 2018Analysis of Hand Segmentation in the Wildcited by 18
  25. 2019Contextual Attention for Hand Detection in the Wildcited by 16
  26. 2024FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotationcited by 12
  27. 2024HOIST-Former: Hand-held Objects Identification, Segmentation, and Tracking in the Wildcited by 9
  28. 2024CaRe-Ego: Contact-aware Relationship Modeling for Egocentric Interactive Hand-object Segmentationcited by 7
  29. 2023Mitigating Perspective Distortion-induced Shape Ambiguity in Image Cropscited by 4
  30. 2026Interaction-aware Representation Modeling with Co-occurrence Consistency for Egocentric Hand-Object Parsingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026GraspHOI: Full-Body 3D Human-Object Reconstruction with Finger-Level Grasps from a Single In-the-Wild Imagecited by 0