mlsanityData to 14 Oct 2026

ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

arXiv 2204.13662 ↗First posted 28 Apr 2022cs.CVcited by 312

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC -- a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de.

Key facts. Cited by 312 papers in the map (data through 14 Oct 2026). Builds on 58 of its 92 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

58 of 92 references.

References · 58

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  7. 2017End-to-end Recovery of Human Shape and Posecited by 885
  8. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  9. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  10. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  11. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  12. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
38 more
  1. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  2. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  3. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  4. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  5. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  6. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
  7. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  8. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  9. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  10. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  11. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  12. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  13. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  14. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 183
  15. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  16. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  17. 2019Category-Level Articulated Object Pose Estimationcited by 159
  18. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  19. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  20. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 144
  21. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 141
  22. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  23. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  24. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 139
  25. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 135
  26. 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 128
  27. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  28. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  29. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  30. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 113
  31. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  32. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  33. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  34. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  35. 2018Cross-modal Deep Variational Hand Pose Estimationcited by 89
  36. 2022Interacting Attention Graph for Single Image Two-Hand Reconstructioncited by 87
  37. 2017Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensorcited by 84
  38. 2016Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Inputcited by 56

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 37; 2024: 70; 2025: 83; 2026: 120.

312 papers.

Sort:

By area:

  1. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 311
  2. 2023Object Motion Guided Human Motion Synthesiscited by 212
  3. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  4. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  5. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
  6. 2023HUGS: Human Gaussian Splatscited by 154
  7. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 129
  8. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  9. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 124
  10. 2023PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objectscited by 118
  11. 2023Controllable Human-Object Interaction Synthesiscited by 109
  12. 20233D Human Pose Estimation via Intuitive Physicscited by 103
38 more
  1. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  2. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 93
  3. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 88
  4. 2023Hi4D: 4D Instance Segmentation of Close Human Interactioncited by 87
  5. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 86
  6. 2023An Outlook into the Future of Egocentric Visioncited by 84
  7. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  8. 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 82
  9. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 78
  10. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  11. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 74
  12. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  13. 2025HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videoscited by 66
  14. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  15. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  16. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  17. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 62
  18. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 60
  19. 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 59
  20. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 57
  21. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
  22. 2024Autonomous Character-Scene Interaction Synthesis from Text Instructioncited by 56
  23. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 55
  24. 2024Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Cameracited by 51
  25. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  26. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 49
  27. 2024Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Trackingcited by 48
  28. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 44
  29. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  30. 2024HanDiffuser: Text-to-Image Generation With Realistic Hand Appearancescited by 42
  31. 2024AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimationcited by 42
  32. 2023Detecting Human-Object Contact in Imagescited by 41
  33. 2023Dexterous Functional Graspingcited by 39
  34. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 38
  35. 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Datacited by 37
  36. 2024PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Taskscited by 37
  37. 2025DexMachina: Functional Retargeting for Bimanual Dexterous Manipulationcited by 37
  38. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 36

50 most cited of 312 All 312 (JSON)