mlsanityData to 7 Oct 2026

GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities

arXiv 2412.04244 ↗First posted 5 Dec 2024cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Understanding bimanual human hand activities is a critical problem in AI and robotics. We cannot build large models of bimanual activities because existing datasets lack the scale, coverage of diverse hand activities, and detailed annotations. We introduce GigaHands, a massive annotated dataset capturing 34 hours of bimanual hand activities from 56 subjects and 417 objects, totaling 14k motion clips derived from 183 million frames paired with 84k text annotations. Our markerless capture setup and data acquisition protocol enable fully automatic 3D hand and object estimation while minimizing the effort required for text annotation. The scale and diversity of GigaHands enable broad applications, including text-driven action synthesis, hand motion captioning, and dynamic radiance field reconstruction. Our website are avaliable at https://ivl.cs.brown.edu/research/gigahands.html .

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 97 of its 159 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

97 of 159 references.

References · 97

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
38 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  3. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  5. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  6. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  7. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  8. 2022Human Motion Diffusion Modelcited by 1,062
  9. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  10. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  12. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  13. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  14. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  15. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  16. 2020Rescaling Egocentric Visioncited by 529
  17. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  18. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
  19. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  20. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  21. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  22. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  23. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  24. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  25. 2023Reconstructing Hands in 3D with Transformerscited by 298
  26. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  27. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  28. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  29. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  30. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  31. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  32. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  33. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  34. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  35. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  36. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  37. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  38. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177

50 most cited of 97 All 97 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 18.

32 papers.

Sort:

By area:

  1. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  2. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 25
  3. 2025OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Modelcited by 17
  4. 2025Dexterous World Modelscited by 15
  5. 2025SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarioscited by 11
  6. 2025OPENTOUCH: Bringing Full-Hand Touch to Real-World Interactioncited by 11
  7. 20252HandedAfforder: Learning Precise Actionable Bimanual Affordances from Human Videoscited by 9
  8. 2025DyTact: Capturing Dynamic Contacts in Hand-Object Manipulationcited by 8
  9. 2025TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactionscited by 8
  10. 2026Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstructioncited by 8
  11. 2025OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfercited by 6
  12. 2026SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wildcited by 4
20 more
  1. 2026PAM: A Pose-Appearance-Motion Engine for Sim-to-Real HOI Video Generationcited by 3
  2. 2026HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesiscited by 3
  3. 2025The Potential and Limitations of Vision-Language Models for Human Motion Understanding: A Case Study in Data-Driven Stroke Rehabilitationcited by 2
  4. 2026NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Datacited by 2
  5. 2026HandX: Scaling Bimanual Motion and Interaction Generationcited by 2
  6. 2026HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodimentscited by 2
  7. 2025SyncMV4D: Synchronized Multi-view Joint Diffusion of Appearance and Motion for Hand-Object Interaction Synthesiscited by 1
  8. 2026CLUTCH: Contextualized Language model for Unlocking Text-Conditioned Hand motion modelling in the wildcited by 1
  9. 2026AnyHand: A Large-Scale Synthetic Dataset for RGB(-D) Hand Pose Estimationcited by 1
  10. 2025BG-HOP: A Bimanual Generative Hand-Object Priorcited by 0
  11. 2025Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rigcited by 0
  12. 2026DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulationcited by 0
  13. 2026DynaHOI: Benchmarking Hand-Object Interaction for Dynamic Targetcited by 0
  14. 2026High-Fidelity 4D Hand-Object Capture via Multi-View Spatiotemporal Tracking and Physics-Aware Gaussianscited by 0
  15. 2026VEPHand: View-Efficient Photometric Hand Performance Capture at Scalecited by 0
  16. 2026A Synthetic-Driven Vision System for Assembly Step Recognitioncited by 0
  17. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 0
  18. 2026ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Enginecited by 0
  19. 2026AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learningcited by 0
  20. 2026TransHands: Repurposing Human Pose Encoders as Hand Pose Encoderscited by 0