mlsanityData to 14 Oct 2026

SAGA: Stochastic Whole-Body Grasping with Contact

arXiv 2112.10103 ↗First posted 19 Dec 2021cs.CVcited by 91

Not on the map. Why?

Abstract (arXiv)

The synthesis of human grasping has numerous applications including AR/VR, video games and robotics. While methods have been proposed to generate realistic hand-object interaction for object grasping and manipulation, these typically only consider interacting hand alone. Our goal is to synthesize whole-body grasping motions. Starting from an arbitrary initial pose, we aim to generate diverse and natural whole-body human motions to approach and grasp a target object in 3D space. This task is challenging as it requires modeling both whole-body dynamics and dexterous finger movements. To this end, we propose SAGA (StochAstic whole-body Grasping with contAct), a framework which consists of two key components: (a) Static whole-body grasping pose generation. Specifically, we propose a multi-task generative model, to jointly learn static whole-body grasping poses and human-object contacts. (b) Grasping motion infilling. Given an initial pose and the generated whole-body grasping pose as the start and end of the motion respectively, we design a novel contact-aware generative motion infilling module to generate a diverse set of grasp-oriented motions. We demonstrate the effectiveness of our method, which is a novel generative framework to synthesize realistic and expressive whole-body motions that approach and grasp randomly placed unseen objects. Code and models are available at https://jiahaoplus.github.io/SAGA/saga.html.

Key facts. Cited by 91 papers in the map (data through 14 Oct 2026). Builds on 39 of its 68 references that are in the map. First posted on arXiv 19 Dec 2021.

Builds on

39 of 68 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  7. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  8. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  9. 2017On human motion prediction using recurrent neural networkscited by 338
  10. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  11. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 298
  12. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
27 more
  1. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  2. 2021Robust Motion In-betweeningcited by 229
  3. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  4. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 185
  5. 2021Character Controllers Using Motion VAEscited by 184
  6. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  7. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  8. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  9. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 147
  10. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 139
  11. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  12. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  13. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  14. 2019Efficient Learning on Point Clouds with Basis Point Setscited by 103
  15. 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
  16. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  17. 2019ContactGrasp: Functional Multi-finger Grasp Synthesis from Contactcited by 87
  18. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  19. 2020Convolutional Autoencoders for Human Motion Infillingcited by 73
  20. 2019Putting Humans in a Scene: Learning Affordance in 3D Indoor Environmentscited by 72
  21. 2018Action-Agnostic Human Pose Forecastingcited by 56
  22. 2017Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learningcited by 52
  23. 2019Imitation Learning for Human Pose Predictioncited by 44
  24. 2021Task-Generic Hierarchical Human Motion Prior using VAEscited by 33
  25. 2017Where and Who? Automatic Semantic-Aware Person Compositioncited by 21
  26. 2019Generating Grasp Poses for a High-DOF Gripper Using Neural Networkscited by 17
  27. 2021Learning to Manipulate Amorphous Materialscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 25; 2024: 31; 2025: 19; 2026: 11.

91 papers.

Sort:

By area:

  1. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  2. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 223
  3. 2023Object Motion Guided Human Motion Synthesiscited by 212
  4. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  5. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  6. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 124
  7. 2023Controllable Human-Object Interaction Synthesiscited by 109
  8. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 98
  9. 2023Human Motion Generation: A Surveycited by 96
  10. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 93
  11. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  12. 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 82
38 more
  1. 2022Full-Body Articulated Human-Object Interactioncited by 78
  2. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  3. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  4. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  5. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  6. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 62
  7. 2024Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitivescited by 61
  8. 2023CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesiscited by 56
  9. 2024Autonomous Character-Scene Interaction Synthesis from Text Instructioncited by 56
  10. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 55
  11. 2023ContactGen: Generative Contact Modeling for Grasp Generationcited by 53
  12. 2024ReGenNet: Towards Human Action-Reaction Synthesiscited by 53
  13. 2023Physically Plausible Full-Body Hand-Object Interaction Synthesiscited by 52
  14. 2023Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environmentscited by 51
  15. 2023Synthesizing Physically Plausible Human Motions in 3D Scenescited by 49
  16. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  17. 2022FLEX: Full-Body Grasping Without Full-Body Graspscited by 46
  18. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  19. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  20. 2023Generative Proxemics: A Prior for 3D Social Interaction from Imagescited by 43
  21. 2023Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Modelscited by 43
  22. 2023Task-Oriented Human-Object Interactions Generation with Implicit Neural Representationscited by 42
  23. 2023Generating Continual Human Motion in Diverse 3D Scenescited by 39
  24. 2023PMP: Learning to Physically Interact with Environments using Part-wise Motion Priorscited by 38
  25. 2023Object pop-up: Can we infer 3D objects and their poses from human interactions alone?cited by 37
  26. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 36
  27. 2024GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusioncited by 35
  28. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  29. 2024RealDex: Towards Human-like Grasping for Robotic Dexterous Handcited by 34
  30. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  31. 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
  32. 2024SemGrasp: Semantic Grasp Generation via Language Aligned Discretizationcited by 29
  33. 2023Revisit Human-Scene Interaction via Space Occupancycited by 25
  34. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  35. 2025A Survey on Human Interaction Motion Generationcited by 25
  36. 2025Guiding Human-Object Interactions with Rich Geometry and Relationscited by 24
  37. 2024GEARS: Local Geometry-aware Hand-object Interaction Synthesiscited by 20
  38. 2024TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactionscited by 20

50 most cited of 91 All 91 (JSON)