# FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras

arXiv: https://arxiv.org/abs/1610.09534 · HTML: https://mlsanity.com/paper/1610.09534/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2016.

## Built on by (most cited first, 23 of 23)

- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time](https://mlsanity.com/paper/2202.08614/) (2022)
- [Editable Free-viewpoint Video Using a Layered Neural Representation](https://mlsanity.com/paper/2104.14786/) (2021)
- [A Survey of Non-Rigid 3D Registration](https://mlsanity.com/paper/2203.07858/) (2022)
- [SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments](https://mlsanity.com/paper/2303.09095/) (2023)
- [NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions](https://mlsanity.com/paper/2202.12825/) (2022)
- [LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors](https://mlsanity.com/paper/2205.15410/) (2022)
- [NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering using RGB Cameras](https://mlsanity.com/paper/2103.07700/) (2021)
- [HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR](https://mlsanity.com/paper/2203.09215/) (2022)
- [Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions](https://mlsanity.com/paper/2108.00362/) (2021)
- [ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References](https://mlsanity.com/paper/2103.06747/) (2021)
- [TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors](https://mlsanity.com/paper/2105.04605/) (2021)
- [HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions](https://mlsanity.com/paper/2203.09287/) (2022)
- [Proactive Multi-Camera Collaboration For 3D Human Pose Estimation](https://mlsanity.com/paper/2303.03767/) (2023)
- [AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2201.08093/) (2022)
- [Human-M3: A Multi-view Multi-modal Dataset for 3D Human Pose Estimation in Outdoor Scenes](https://mlsanity.com/paper/2308.00628/) (2023)
- [Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery](https://mlsanity.com/paper/2301.06020/) (2023)
- [LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environment](https://mlsanity.com/paper/2407.09833/) (2024)
- [HiSC4D: Human-centered interaction and 4D Scene Capture in Large-scale Space Using Wearable IMUs and LiDAR](https://mlsanity.com/paper/2409.04398/) (2024)
- [3D Human Reconstruction in the Wild with Collaborative Aerial Cameras](https://mlsanity.com/paper/2108.03936/) (2021)
- [Markerless Body Motion Capturing for 3D Character Animation based on Multi-view Cameras](https://mlsanity.com/paper/2212.05788/) (2022)
- [SPARE: Symmetrized Point-to-Plane Distance for Robust Non-Rigid 3D Registration](https://mlsanity.com/paper/2405.20188/) (2024)
- [B'ezier Degradation Modeling for LiDAR-based Human Motion Capture](https://mlsanity.com/paper/2605.19620/) (2026)

## Abstract (arXiv)

Aiming at automatic, convenient and non-instrusive motion capture, this paper presents a new generation markerless motion capture technique, the FlyCap system, to capture surface motions of moving characters using multiple autonomous flying cameras (autonomous unmanned aerial vehicles(UAV) each integrated with an RGBD video camera). During data capture, three cooperative flying cameras automatically track and follow the moving target who performs large scale motions in a wide space. We propose a novel non-rigid surface registration method to track and fuse the depth of the three flying cameras for surface motion tracking of the moving target, and simultaneously calculate the pose of each flying camera. We leverage the using of visual-odometry information provided by the UAV platform, and formulate the surface tracking problem in a non-linear objective function that can be linearized and effectively minimized through a Gaussian-Newton method. Quantitative and qualitative experimental results demonstrate the competent and plausible surface and motion reconstruction results