# SmartDepthSync: Open Source Synchronized Video Recording System of Smartphone RGB and Depth Camera Range Image Frames with Sub-millisecond Precision

arXiv: https://arxiv.org/abs/2111.03552 · HTML: https://mlsanity.com/paper/2111.03552/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 5 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [The TUM VI Benchmark for Evaluating Visual-Inertial Odometry](https://mlsanity.com/paper/1804.06120/) (2018)
- [Tightly Coupled 3D Lidar Inertial Odometry and Mapping](https://mlsanity.com/paper/1904.06993/) (2019)
- [ADVIO: An authentic dataset for visual-inertial odometry](https://mlsanity.com/paper/1807.09828/) (2018)
- [Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08630/) (2021)
- [Wireless Software Synchronization of Multiple Distributed Cameras](https://mlsanity.com/paper/1812.09366/) (2018)
- [Stereo Visual Inertial LiDAR Simultaneous Localization and Mapping](https://mlsanity.com/paper/1902.10741/) (2019)
- [VersaVIS: An Open Versatile Multi-Camera Visual-Inertial Sensor Suite](https://mlsanity.com/paper/1912.02469/) (2019)
- [The Matter of Time -- A General and Efficient System for Precise Sensor Synchronization in Robotic Computing](https://mlsanity.com/paper/2103.16045/) (2021)
- [Rolling Shutter Camera Synchronization with Sub-millisecond Accuracy](https://mlsanity.com/paper/1902.11084/) (2019)
- [Open-Source LiDAR Time Synchronization System by Mimicking GNSS-clock](https://mlsanity.com/paper/2107.02625/) (2021)

## Abstract (arXiv)

Nowadays, smartphones can produce a synchronized (synced) stream of high-quality data, including RGB images, inertial measurements, and other data. Therefore, smartphones are becoming appealing sensor systems in the robotics community. Unfortunately, there is still the need for external supporting sensing hardware, such as a depth camera precisely synced with the smartphone sensors. In this paper, we propose a hardware-software recording system that presents a heterogeneous structure and contains a smartphone and an external depth camera for recording visual, depth, and inertial data that are mutually synchronized. The system is synced at the time and the frame levels: every RGB image frame from the smartphone camera is exposed at the same moment of time with a depth camera frame with sub-millisecond precision. We provide a method and a tool for sync performance evaluation that can be applied to any pair of depth and RGB cameras. Our system could be replicated, modified, or extended by employing our open-sourced materials.