# Multi-view data capture for dynamic object reconstruction using handheld augmented reality mobiles

arXiv: https://arxiv.org/abs/2103.07883 · HTML: https://mlsanity.com/paper/2103.07883/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 49 references that are in the map.
- First posted on arXiv 14 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [Soccer on Your Tabletop](https://mlsanity.com/paper/1806.00890/) (2018)
- [Wireless Software Synchronization of Multiple Distributed Cameras](https://mlsanity.com/paper/1812.09366/) (2018)
- [Temporally Coherent General Dynamic Scene Reconstruction](https://mlsanity.com/paper/1907.08195/) (2019)
- [Dense Wide-Baseline Scene Flow From Two Handheld Video Cameras](https://mlsanity.com/paper/1609.05115/) (2016)
- [U4D: Unsupervised 4D Dynamic Scene Understanding](https://mlsanity.com/paper/1907.09905/) (2019)
- [CamSwarm: Instantaneous Smartphone Camera Arrays for Collaborative Photography](https://mlsanity.com/paper/1507.01148/) (2015)
- [Multi-view data capture using edge-synchronised mobiles](https://mlsanity.com/paper/2005.03286/) (2020)

## Built on by (most cited first, 2 of 2)

- [Sub-millisecond Video Synchronization of Multiple Android Smartphones](https://mlsanity.com/paper/2107.00987/) (2021)
- [When Simultaneous Localization and Mapping Meets Wireless Communications: A Survey](https://mlsanity.com/paper/2602.06995/) (2026)

## Abstract (arXiv)

We propose a system to capture nearly-synchronous frame streams from multiple and moving handheld mobiles that is suitable for dynamic object 3D reconstruction. Each mobile executes Simultaneous Localisation and Mapping on-board to estimate its pose, and uses a wireless communication channel to send or receive synchronisation triggers. Our system can harvest frames and mobile poses in real time using a decentralised triggering strategy and a data-relay architecture that can be deployed either at the Edge or in the Cloud. We show the effectiveness of our system by employing it for 3D skeleton and volumetric reconstructions. Our triggering strategy achieves equal performance to that of an NTP-based synchronisation approach, but offers higher flexibility, as it can be adjusted online based on application needs. We created a challenging new dataset, namely 4DM, that involves six handheld augmented reality mobiles recording an actor performing sports actions outdoors. We validate our system on 4DM, analyse its strengths and limitations, and compare its modules with alternative ones.