mlsanityData to 7 Oct 2026

Real-time RGBD-based Extended Body Pose Estimation

arXiv 2103.03663 ↗First posted 5 Mar 2021cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

We present a system for real-time RGBD-based estimation of 3D human pose. We use parametric 3D deformable human mesh model (SMPL-X) as a representation and focus on the real-time estimation of parameters for the body pose, hands pose and facial expression from Kinect Azure RGB-D camera. We train estimators of body pose and facial expression parameters. Both estimators use previously published landmark extractors as input and custom annotated datasets for supervision, while hand pose is estimated directly by a previously published method. We combine the predictions of those estimators into a temporally-smooth human pose. We train the facial expression extractor on a large talking face dataset, which we annotate with facial expression parameters. For the body pose we collect and annotate a dataset of 56 people captured from a rig of 5 Kinect Azure RGB-D cameras and use it together with a large motion capture AMASS dataset. Our RGB-D body pose model outperforms the state-of-the-art RGB-only methods and works on the same level of accuracy compared to a slower RGB-D optimization-based solution. The combined system runs at 30 FPS on a server with a single GPU. The code will be available at https://saic-violet.github.io/rgbd-kinect-pose

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

36 of 54 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2015Fast R-CNNcited by 2,930
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  6. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  7. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  8. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  9. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  10. 2017End-to-end Recovery of Human Shape and Posecited by 883
  11. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  12. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
24 more
  1. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  2. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  3. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  4. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  5. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  6. 2017Integral Human Pose Regressioncited by 255
  7. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  8. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  9. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  10. 2019Learnable Triangulation of Human Posecited by 160
  11. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  12. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  13. 2017Compositional Human Pose Regressioncited by 124
  14. 2020Skeleton-Aware Networks for Deep Motion Retargetingcited by 118
  15. 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
  16. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  17. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  18. 2019Structured Prediction Helps 3D Human Motion Modellingcited by 96
  19. 2020Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flowscited by 94
  20. 2019Real-time Facial Surface Geometry from Monocular Video on Mobile GPUscited by 72
  21. 2020Monocular Human Pose Estimation: A Survey of Deep Learning-based Methodscited by 59
  22. 2019Learning Character-Agnostic Motion for Motion Retargeting in 2Dcited by 44
  23. 2016Deep Kinematic Pose Regressioncited by 40
  24. 2020Residual Pose: A Decoupled Approach for Depth-based 3D Human Pose Estimationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 3; 2026: 2.

12 papers.

  1. 2023DINAR: Diffusion Inpainting of Neural Textures for One-Shot Human Avatarscited by 28
  2. 2024HMD-Poser: On-Device Real-time Human Motion Tracking from Scalable Sparse Observationscited by 22
  3. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  4. 2023PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Cloudscited by 9
  5. 2025GA3CE: Unconstrained 3D Gaze Estimation with Gaze-Aware 3D Context Encodingcited by 3
  6. 2024Sports Analysis and VR Viewing System Based on Player Tracking and Pose Estimation with Multimodal and Multiview Sensorscited by 2
  7. 2023LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-timestamp 3D Human Pose Estimationcited by 1
  8. 2024D-PoSE: Depth as an Intermediate Representation for 3D Human Pose and Shape Estimationcited by 1
  9. 2025Age-Inclusive 3D Human Mesh Recovery for Action-Preserving Data Anonymizationcited by 1
  10. 2026Stereo-Inertial Poser: Towards Metric-Accurate Shape-Aware Motion Capture Using Sparse IMUs and a Single Stereo Cameracited by 1
  11. 20253D Human Mesh Estimation from Single View RGBDcited by 0
  12. 2026B'ezier Degradation Modeling for LiDAR-based Human Motion Capturecited by 0