# 3D Human Reconstruction in the Wild with Collaborative Aerial Cameras

arXiv: https://arxiv.org/abs/2108.03936 · HTML: https://mlsanity.com/paper/2108.03936/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 22 of 22)

- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras](https://mlsanity.com/paper/1610.09534/) (2016)
- [Towards a Robust Aerial Cinematography Platform: Localizing and Tracking Moving Targets in Unstructured Environments](https://mlsanity.com/paper/1904.02319/) (2019)
- [Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Making](https://mlsanity.com/paper/1910.06988/) (2019)
- [AirCapRL: Autonomous Aerial Human Motion Capture using Deep Reinforcement Learning](https://mlsanity.com/paper/2007.06343/) (2020)
- [Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography](https://mlsanity.com/paper/1904.02579/) (2019)
- [Active Perception based Formation Control for Multiple Aerial Vehicles](https://mlsanity.com/paper/1901.07813/) (2019)
- [Human Motion Capture Using a Drone](https://mlsanity.com/paper/1804.06112/) (2018)
- [Multiview Supervision By Registration](https://mlsanity.com/paper/1811.11251/) (2018)
- [Autonomous drone cinematographer: Using artistic principles to create smooth, safe, occlusion-free trajectories for aerial filming](https://mlsanity.com/paper/1808.09563/) (2018)
- [Optimizing for Aesthetically Pleasing Quadrotor Camera Motion](https://mlsanity.com/paper/1906.11686/) (2019)
- [Optimal Trajectory Planning for Cinematography with Multiple Unmanned Aerial Vehicles](https://mlsanity.com/paper/2009.04234/) (2020)
- [Do You See What I See? Coordinating Multiple Aerial Cameras for Robot Cinematography](https://mlsanity.com/paper/2011.05437/) (2020)
- [Airways: Optimization-Based Planning of Quadrotor Trajectories according to High-Level User Goals](https://mlsanity.com/paper/1906.11669/) (2019)
- [Autonomous Execution of Cinematographic Shots with Multiple Drones](https://mlsanity.com/paper/2006.12163/) (2020)
- [Active localization of multiple targets using noisy relative measurements](https://mlsanity.com/paper/2002.09850/) (2020)
- [Autonomous Planning for Multiple Aerial Cinematographers](https://mlsanity.com/paper/2005.07237/) (2020)
- [End-to-End 3D Multi-Object Tracking and Trajectory Forecasting](https://mlsanity.com/paper/2008.11598/) (2020)

## Built on by (most cited first, 4 of 4)

- [Proactive Multi-Camera Collaboration For 3D Human Pose Estimation](https://mlsanity.com/paper/2303.03767/) (2023)
- [Greedy Perspectives: Multi-Drone View Planning for Collaborative Perception in Cluttered Environments](https://mlsanity.com/paper/2310.10863/) (2023)
- [PoseSyn: Synthesizing Diverse 3D Pose Data from In-the-Wild 2D Data](https://mlsanity.com/paper/2503.13025/) (2025)
- [Real-Time Hybrid Mapping of Populated Indoor Scenes using a Low-Cost Monocular UAV](https://mlsanity.com/paper/2203.02453/) (2022)

## Abstract (arXiv)

Aerial vehicles are revolutionizing applications that require capturing the 3D structure of dynamic targets in the wild, such as sports, medicine, and entertainment. The core challenges in developing a motion-capture system that operates in outdoors environments are: (1) 3D inference requires multiple simultaneous viewpoints of the target, (2) occlusion caused by obstacles is frequent when tracking moving targets, and (3) the camera and vehicle state estimation is noisy. We present a real-time aerial system for multi-camera control that can reconstruct human motions in natural environments without the use of special-purpose markers. We develop a multi-robot coordination scheme that maintains the optimal flight formation for target reconstruction quality amongst obstacles. We provide studies evaluating system performance in simulation, and validate real-world performance using two drones while a target performs activities such as jogging and playing soccer. Supplementary video: https://youtu.be/jxt91vx0cns