# Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows

arXiv: https://arxiv.org/abs/2204.02509 · HTML: https://mlsanity.com/paper/2204.02509/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 49 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Consistent Video Depth Estimation](https://mlsanity.com/paper/2004.15021/) (2020)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [Robust Consistent Video Depth Estimation](https://mlsanity.com/paper/2012.05901/) (2020)
- [DeepV2D: Video to Depth with Differentiable Structure from Motion](https://mlsanity.com/paper/1812.04605/) (2018)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [Towards Long-Form Video Understanding](https://mlsanity.com/paper/2106.11310/) (2021)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [Learning the Depths of Moving People by Watching Frozen People](https://mlsanity.com/paper/1904.11111/) (2019)
- [DeepSFM: Structure From Motion Via Deep Bundle Adjustment](https://mlsanity.com/paper/1912.09697/) (2019)
- [Measuring robustness of Visual SLAM](https://mlsanity.com/paper/1910.04755/) (2019)
- [Triangulation: Why Optimize?](https://mlsanity.com/paper/1907.11917/) (2019)

## Built on by (most cited first, 15 of 15)

- [MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion](https://mlsanity.com/paper/2504.20040/) (2025)
- [EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queries](https://mlsanity.com/paper/2212.06969/) (2022)
- [Relative Pose Estimation through Affine Corrections of Monocular Depth Priors](https://mlsanity.com/paper/2501.05446/) (2025)
- [Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images](https://mlsanity.com/paper/2402.05869/) (2024)
- [RePoseD: Efficient Relative Pose Estimation With Known Depth Information](https://mlsanity.com/paper/2501.07742/) (2025)
- [EC-SfM: Efficient Covisibility-based Structure-from-Motion for Both Sequential and Unordered Images](https://mlsanity.com/paper/2302.10544/) (2023)
- [Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation](https://mlsanity.com/paper/2511.22429/) (2025)
- [ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field](https://mlsanity.com/paper/2401.08140/) (2024)
- [Depth Completion with Multiple Balanced Bases and Confidence for Dense Monocular SLAM](https://mlsanity.com/paper/2309.04145/) (2023)
- [VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion](https://mlsanity.com/paper/2607.27194/) (2026)
- [Framework for 2D Ad placements in LinearTV](https://mlsanity.com/paper/2212.02450/) (2022)
- [Large-scale visual SLAM for in-the-wild videos](https://mlsanity.com/paper/2504.20496/) (2025)
- [Textured Geometry Evaluation: Perceptual 3D Textured Shape Metric via 3D Latent-Geometry Network](https://mlsanity.com/paper/2512.01380/) (2025)
- [Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth](https://mlsanity.com/paper/2605.19797/) (2026)
- [BLASt3R: Bundle Adjustment of Any Image Set with Multi-View Matching and Monocular Priors](https://mlsanity.com/paper/2609.05210/) (2026)

## Abstract (arXiv)

Existing approaches for Structure from Motion (SfM) produce impressive 3-D reconstruction results especially when using imagery captured with large parallax. However, to create engaging video-content in movies and TV shows, the amount by which a camera can be moved while filming a particular shot is often limited. The resulting small-motion parallax between video frames makes standard geometry-based SfM approaches not as effective for movies and TV shows. To address this challenge, we propose a simple yet effective approach that uses single-frame depth-prior obtained from a pretrained network to significantly improve geometry-based SfM for our small-parallax setting. To this end, we first use the depth-estimates of the detected keypoints to reconstruct the point cloud and camera-pose for initial two-view reconstruction. We then perform depth-regularized optimization to register new images and triangulate the new points during incremental reconstruction. To comprehensively evaluate our approach, we introduce a new dataset (StudioSfM) consisting of 130 shots with 21K frames from 15 studio-produced videos that are manually annotated by a professional CG studio. We demonstrate that our approach: (a) significantly improves the quality of 3-D reconstruction for our small-parallax setting, (b) does not cause any degradation for data with large-parallax, and (c) maintains the generalizability and scalability of geometry-based sparse SfM. Our dataset can be obtained at https://github.com/amazon-research/small-baseline-camera-tracking.