# H2O: A Benchmark for Visual Human-human Object Handover Analysis

arXiv: https://arxiv.org/abs/2104.11466 · HTML: https://mlsanity.com/paper/2104.11466/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions](https://mlsanity.com/paper/1904.05349/) (2019)
- [Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction](https://mlsanity.com/paper/2004.13449/) (2020)
- [CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction](https://mlsanity.com/paper/2012.00924/) (2020)
- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis](https://mlsanity.com/paper/1704.02612/) (2017)
- [BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks](https://mlsanity.com/paper/2008.05079/) (2020)
- [Learning Dynamic Robot-to-Human Object Handover from Human Feedback](https://mlsanity.com/paper/1603.06390/) (2016)
- [Human Grasp Classification for Reactive Human-to-Robot Handovers](https://mlsanity.com/paper/2003.06000/) (2020)
- [Human Detection and Tracking for Video Surveillance A Cognitive Science Approach](https://mlsanity.com/paper/1709.00726/) (2017)

## Built on by (most cited first, 16 of 16)

- [OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction](https://mlsanity.com/paper/2203.15709/) (2022)
- [TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding](https://mlsanity.com/paper/2401.08399/) (2024)
- [AffordPose: A Large-scale Dataset of Hand-Object Interactions with Affordance-driven Hand Pose](https://mlsanity.com/paper/2309.08942/) (2023)
- [CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement](https://mlsanity.com/paper/2406.19353/) (2024)
- [HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data](https://mlsanity.com/paper/2403.12011/) (2024)
- [Learning Human-to-Robot Handovers from Point Clouds](https://mlsanity.com/paper/2303.17592/) (2023)
- [GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities](https://mlsanity.com/paper/2412.04244/) (2024)
- [RenderIH: A Large-scale Synthetic Dataset for 3D Interacting Hand Pose Estimation](https://mlsanity.com/paper/2309.09301/) (2023)
- [HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers](https://mlsanity.com/paper/2205.09747/) (2022)
- [SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers](https://mlsanity.com/paper/2311.05599/) (2023)
- [MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints](https://mlsanity.com/paper/2404.10227/) (2024)
- [HOH: Markerless Multimodal Human-Object-Human Handover Dataset with Large Object Count](https://mlsanity.com/paper/2310.00723/) (2023)
- [Ins-HOI: Instance Aware Human-Object Interactions Recovery](https://mlsanity.com/paper/2312.09641/) (2023)
- [ContactHandover: Contact-Guided Robot-to-Human Object Handover](https://mlsanity.com/paper/2404.01402/) (2024)
- [3HANDS Dataset: Learning from Humans for Generating Naturalistic Handovers with Supernumerary Robotic Limbs](https://mlsanity.com/paper/2503.04635/) (2025)
- [RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction](https://mlsanity.com/paper/2608.13028/) (2026)

## Abstract (arXiv)

Object handover is a common human collaboration behavior that attracts attention from researchers in Robotics and Cognitive Science. Though visual perception plays an important role in the object handover task, the whole handover process has been specifically explored. In this work, we propose a novel rich-annotated dataset, H2O, for visual analysis of human-human object handovers. The H2O, which contains 18K video clips involving 15 people who hand over 30 objects to each other, is a multi-purpose benchmark. It can support several vision-based tasks, from which, we specifically provide a baseline method, RGPNet, for a less-explored task named Receiver Grasp Prediction. Extensive experiments show that the RGPNet can produce plausible grasps based on the giver's hand-object states in the pre-handover phase. Besides, we also report the hand and object pose errors with existing baselines and show that the dataset can serve as the video demonstrations for robot imitation learning on the handover task. Dataset, model and code will be made public.