mlsanityData to 7 Oct 2026

DivaTrack: Diverse Bodies and Motions from Acceleration-Enhanced Three-Point Trackers

arXiv 2402.09211 ↗First posted 14 Feb 2024cs.CV cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Full-body avatar presence is crucial for immersive social and environmental interactions in digital reality. However, current devices only provide three six degrees of freedom (DOF) poses from the headset and two controllers (i.e. three-point trackers). Because it is a highly under-constrained problem, inferring full-body pose from these inputs is challenging, especially when supporting the full range of body proportions and use cases represented by the general population. In this paper, we propose a deep learning framework, DivaTrack, which outperforms existing methods when applied to diverse body sizes and activities. We augment the sparse three-point inputs with linear accelerations from Inertial Measurement Units (IMU) to improve foot contact prediction. We then condition the otherwise ambiguous lower-body pose with the predictions of foot contact and upper-body pose in a two-stage model. We further stabilize the inferred full-body pose in a wide range of configurations by learning to blend predictions that are computed in two reference frames, each of which is designed for different types of motions. We demonstrate the effectiveness of our design on a large dataset that captures 22 subjects performing challenging locomotion for three-point tracking, including lunges, hula-hooping, and sitting. As shown in a live demo using the Meta VR headset and Xsens IMUs, our method runs in real-time while accurately tracking a user's motion when they perform a diverse set of movements.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 47 of its 80 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

47 of 80 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2018SlowFast Networks for Video Recognitioncited by 1,862
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  7. 2022Human Motion Diffusion Modelcited by 1,062
  8. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  9. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  10. 2023Human Motion Diffusion as a Generative Priorcited by 354
  11. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  12. 2018Learning 3D Human Dynamics from Videocited by 315
35 more
  1. 2022EDGE: Editable Dance Generation From Musiccited by 284
  2. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  3. 2021Character Controllers Using Motion VAEscited by 182
  4. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 168
  5. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  6. 2022Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensorscited by 144
  7. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 140
  8. 2021Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensorscited by 137
  9. 2017Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUscited by 108
  10. 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 107
  11. 2023Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Modelcited by 104
  12. 2021Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimationcited by 101
  13. 2022Ego-Body Pose Estimation via Ego-Head Pose Estimationcited by 93
  14. 2020Learning to Generate Diverse Dance Motions with Transformercited by 85
  15. 2022Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldscited by 85
  16. 2022Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generationcited by 75
  17. 2022QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatarscited by 74
  18. 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 58
  19. 2022UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capturecited by 56
  20. 2022FLAG: Flow-based 3D Avatar Generation from Sparse Observationscited by 54
  21. 2023IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbudscited by 52
  22. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  23. 2020SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Cameracited by 49
  24. 2023EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensorscited by 48
  25. 2018Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Timecited by 47
  26. 2019MoGlow: Probabilistic and controllable motion synthesis using normalising flowscited by 47
  27. 2021LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signalscited by 45
  28. 2018Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Controlcited by 39
  29. 2023EgoPoser: Robust Real-Time Egocentric Pose Estimation from Sparse and Intermittent Observations Everywherecited by 31
  30. 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
  31. 2023QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensorscited by 26
  32. 2022Neural3Points: Learning to Generate Physically Realistic Full-body Motion for Virtual Reality Userscited by 20
  33. 2023Scene-Aware 3D Multi-Human Motion Capture from a Single Cameracited by 19
  34. 2022Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer-Grade VR Devicescited by 13
  35. 2019Towards Robust Direction Invariance in Character Animationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 5.

11 papers.

  1. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
  2. 2024HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Devicecited by 26
  3. 2025Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Inputcited by 17
  4. 2024EgoHDM: An Online Egocentric-Inertial Human Motion Capture, Localization, and Dense Mapping Systemcited by 9
  5. 2025Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokenscited by 4
  6. 2026NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Datacited by 2
  7. 2025Step2Motion: Locomotion Reconstruction from Pressure Sensing Insolescited by 0
  8. 2026Garment Inertial Denoiser (GID): Endowing Accurate Motion Capture via Loose IMU Denoisercited by 0
  9. 2026EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversalcited by 0
  10. 2026One Sensor, Whole Body - 3D Body Pose from a Single Consumer Earbud IMUcited by 0
  11. 2026Reliability-Gated Fusion of Consumer Head and Foot IMUs for Lower-Body 3D Posecited by 0