# RoMo: A Robust Solver for Full-body Unlabeled Optical Motion Capture

arXiv: https://arxiv.org/abs/2410.02788 · HTML: https://mlsanity.com/paper/2410.02788/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Builds on (most cited first, 17 of 17)

- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor](https://mlsanity.com/paper/1504.02340/) (2015)
- [SOMA: Solving Optical Marker-Based MoCap Automatically](https://mlsanity.com/paper/2110.04431/) (2021)
- [Transformers in 3D Point Clouds: A Survey](https://mlsanity.com/paper/2205.07417/) (2022)
- [Part Aware Contrastive Learning for Self-Supervised Action Recognition](https://mlsanity.com/paper/2305.00666/) (2023)
- [End-to-End Learning Deep CRF models for Multi-Object Tracking](https://mlsanity.com/paper/1907.12176/) (2019)
- [Auto-labelling of Markers in Optical Motion Capture by Permutation Learning](https://mlsanity.com/paper/1907.13580/) (2019)

## Built on by (most cited first, 6 of 6)

- [StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Data](https://mlsanity.com/paper/2505.03154/) (2025)
- [OpenMoCap: Rethinking Optical Motion Capture under Real-world Occlusion](https://mlsanity.com/paper/2508.12610/) (2025)
- [Mocap Anywhere: Towards Pairwise-Distance based Motion Capture in the Wild (for the Wild)](https://mlsanity.com/paper/2601.19519/) (2026)
- [EMA: Effort Metric Attention for Anatomical Effort-Guided Human Motion Diffusion](https://mlsanity.com/paper/2605.24566/) (2026)
- [DirtyMoCap: Robust Motion Capture from Unconstrained Markers](https://mlsanity.com/paper/2609.19927/) (2026)
- [CAST: Real-Time Motion Capture for Any Skeleton Topology](https://mlsanity.com/paper/2610.14856/) (2026)

## Abstract (arXiv)

Optical motion capture (MoCap) is the "gold standard" for accurately capturing full-body motions. To make use of raw MoCap point data, the system labels the points with corresponding body part locations and solves the full-body motions. However, MoCap data often contains mislabeling, occlusion and positional errors, requiring extensive manual correction. To alleviate this burden, we introduce RoMo, a learning-based framework for robustly labeling and solving raw optical motion capture data. In the labeling stage, RoMo employs a divide-and-conquer strategy to break down the complex full-body labeling challenge into manageable subtasks: alignment, full-body segmentation and part-specific labeling. To utilize the temporal continuity of markers, RoMo generates marker tracklets using a K-partite graph-based clustering algorithm, where markers serve as nodes, and edges are formed based on positional and feature similarities. For motion solving, to prevent error accumulation along the kinematic chain, we introduce a hybrid inverse kinematic solver that utilizes joint positions as intermediate representations and adjusts the template skeleton to match estimated joint positions. We demonstrate that RoMo achieves high labeling and solving accuracy across multiple metrics and various datasets. Extensive comparisons show that our method outperforms state-of-the-art research methods. On a real dataset, RoMo improves the F1 score of hand labeling from 0.94 to 0.98, and reduces joint position error of body motion solving by 25%. Furthermore, RoMo can be applied in scenarios where commercial systems are inadequate. The code and data for RoMo are available at https://github.com/non-void/RoMo.