mlsanityData to 14 Oct 2026

Noise-in, Bias-out: Balanced and Real-time MoCap Solving

arXiv 2309.14330 ↗First posted 25 Sep 2023cs.CV cs.GR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Real-time optical Motion Capture (MoCap) systems have not benefited from the advances in modern data-driven modeling. In this work we apply machine learning to solve noisy unstructured marker estimates in real-time and deliver robust marker-based MoCap even when using sparse affordable sensors. To achieve this we focus on a number of challenges related to model training, namely the sourcing of training data and their long-tailed distribution. Leveraging representation learning we design a technique for imbalanced regression that requires no additional data or labels and improves the performance of our model in rare and challenging poses. By relying on a unified representation, we show that training such a model is not bound to high-end MoCap training data acquisition, and exploit the advances in marker-less MoCap to acquire the necessary data. Finally, we take a step towards richer and affordable MoCap by adapting a body model-based inverse kinematics solution to account for measurement and inference uncertainty, further improving performance and robustness. Project page: https://moverseai.github.io/noise-tail

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 93 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

45 of 93 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  8. 2015Variational Inference with Normalizing Flowscited by 2,170
  9. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  10. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  11. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  12. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
33 more
  1. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  2. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  3. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  4. 2017Learning from Synthetic Humanscited by 359
  5. 2017Integral Human Pose Regressioncited by 256
  6. 2019On the "steerability" of generative adversarial networkscited by 248
  7. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246
  8. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  9. 2017A General and Adaptive Robust Loss Functioncited by 236
  10. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  11. 2019Distribution-Aware Coordinate Representation for Human Pose Estimationcited by 165
  12. 2019Learnable Triangulation of Human Posecited by 161
  13. 2021Delving into Deep Imbalanced Regressioncited by 158
  14. 2021imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Posecited by 103
  15. 2022Balanced MSE for Imbalanced Visual Regressioncited by 89
  16. 2022Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldscited by 85
  17. 2018Numerical Coordinate Regression with Convolutional Neural Networkscited by 72
  18. 2017Human Pose Regression by Combining Indirect Part Detection and Contextual Informationcited by 52
  19. 2022RankSim: Ranking Similarity Regularization for Deep Imbalanced Regressioncited by 52
  20. 2022Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesiscited by 40
  21. 2022Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projectioncited by 39
  22. 2021Adversarial Parametric Pose Priorcited by 38
  23. 2020Perpetual Motion: Generating Unbounded Human Motioncited by 36
  24. 2021SOMA: Solving Optical Marker-Based MoCap Automaticallycited by 31
  25. 2022SUPR: A Sparse Unified Part-Based Human Representationcited by 21
  26. 20183D Human Pose Estimation with 2D Marginal Heatmapscited by 20
  27. 2019Learning to Train with Synthetic Humanscited by 14
  28. 2016UBL: an R package for Utility-based Learningcited by 12
  29. 2020Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memorycited by 11
  30. 2021UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Modelcited by 9
  31. 2022Model Optimization in Imbalanced Regressioncited by 7
  32. 2020It Is Likely That Your Loss Should be a Likelihoodcited by 5
  33. 2019Auto-labelling of Markers in Optical Motion Capture by Permutation Learningcited by 4

Built on by

Not cited yet.