# An Integrated Platform for Live 3D Human Reconstruction and Motion Capturing

arXiv: https://arxiv.org/abs/1712.03084 · HTML: https://mlsanity.com/paper/1712.03084/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2017.

## Built on by (most cited first, 12 of 12)

- [HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media](https://mlsanity.com/paper/2110.07235/) (2021)
- [High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos](https://mlsanity.com/paper/2311.01214/) (2023)
- [Headset: Human emotion awareness under partial occlusions multimodal dataset](https://mlsanity.com/paper/2402.09107/) (2024)
- [SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wild](https://mlsanity.com/paper/2603.28760/) (2026)
- [OMEGAS: Object Mesh Extraction from Large Scenes Guided by Gaussian Segmentation](https://mlsanity.com/paper/2404.15891/) (2024)
- [Simultaneously Recovering Multi-Person Meshes and Multi-View Cameras with Human Semantics](https://mlsanity.com/paper/2412.18785/) (2024)
- [Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig](https://mlsanity.com/paper/2510.02601/) (2025)
- [YOWO: You Only Walk Once to Jointly Map An Indoor Scene and Register Ceiling-mounted Cameras](https://mlsanity.com/paper/2511.16521/) (2025)
- [DeepMoCap: Deep Optical Motion Capture Using Multiple Depth Sensors and Retro-Reflectors](https://mlsanity.com/paper/2110.07283/) (2021)
- [GMFL-Net: A Global Multi-geometric Feature Learning Network for Repetitive Action Counting](https://mlsanity.com/paper/2409.00330/) (2024)
- [DynaTok: Token-Based 4D Reconstruction from Partial Point Clouds](https://mlsanity.com/paper/2606.12189/) (2026)
- [Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstruction](https://mlsanity.com/paper/2607.13646/) (2026)

## Abstract (arXiv)

The latest developments in 3D capturing, processing, and rendering provide means to unlock novel 3D application pathways. The main elements of an integrated platform, which target tele-immersion and future 3D applications, are described in this paper, addressing the tasks of real-time capturing, robust 3D human shape/appearance reconstruction, and skeleton-based motion tracking. More specifically, initially, the details of a multiple RGB-depth (RGB-D) capturing system are given, along with a novel sensors' calibration method. A robust, fast reconstruction method from multiple RGB-D streams is then proposed, based on an enhanced variation of the volumetric Fourier transform-based method, parallelized on the Graphics Processing Unit, and accompanied with an appropriate texture-mapping algorithm. On top of that, given the lack of relevant objective evaluation methods, a novel framework is proposed for the quantitative evaluation of real-time 3D reconstruction systems. Finally, a generic, multiple depth stream-based method for accurate real-time human skeleton tracking is proposed. Detailed experimental results with multi-Kinect2 data sets verify the validity of our arguments and the effectiveness of the proposed system and methodologies.