# Monocular Dense 3D Reconstruction of a Complex Dynamic Scene from Two Perspective Frames

arXiv: https://arxiv.org/abs/1708.04398 · HTML: https://mlsanity.com/paper/1708.04398/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2017.

## Built on by (most cited first, 27 of 27)

- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Shape of Motion: 4D Reconstruction from a Single Video](https://mlsanity.com/paper/2407.13764/) (2024)
- [Robust Dynamic Radiance Fields](https://mlsanity.com/paper/2301.02239/) (2023)
- [Neural Trajectory Fields for Dynamic Novel View Synthesis](https://mlsanity.com/paper/2105.05994/) (2021)
- [VA-DepthNet: A Variational Approach to Single Image Depth Prediction](https://mlsanity.com/paper/2302.06556/) (2023)
- [Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction](https://mlsanity.com/paper/2503.16318/) (2025)
- [Any4D: Unified Feed-Forward Metric 4D Reconstruction](https://mlsanity.com/paper/2512.10935/) (2025)
- [Uncertainty-Aware Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2202.13071/) (2022)
- [Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2110.05594/) (2021)
- [Organic Priors in Non-Rigid Structure from Motion](https://mlsanity.com/paper/2207.06262/) (2022)
- [Single Image Depth Prediction Made Better: A Multivariate Gaussian Take](https://mlsanity.com/paper/2303.18164/) (2023)
- [DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction](https://mlsanity.com/paper/2412.04464/) (2024)
- [Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo](https://mlsanity.com/paper/2110.05621/) (2021)
- [DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images](https://mlsanity.com/paper/2207.02025/) (2022)
- [Enhanced Stable View Synthesis](https://mlsanity.com/paper/2303.17094/) (2023)
- [UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images](https://mlsanity.com/paper/2602.24290/) (2026)
- [Multi-view Monocular Depth and Uncertainty Prediction with Deep SfM in Dynamic Environments](https://mlsanity.com/paper/2201.08633/) (2022)
- [Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior](https://mlsanity.com/paper/2310.11284/) (2023)
- [Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction](https://mlsanity.com/paper/2605.31595/) (2026)
- [Learning Multi-frame and Monocular Prior for Estimating Geometry in Dynamic Scenes](https://mlsanity.com/paper/2505.01737/) (2025)
- [4D Primitive-Mach'e: Glueing Primitives for Persistent 4D Scene Reconstruction](https://mlsanity.com/paper/2512.16564/) (2025)
- [Reconstructing Small 3D Objects in front of a Textured Background](https://mlsanity.com/paper/2105.11352/) (2021)
- [Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion](https://mlsanity.com/paper/2208.03440/) (2022)
- [ZeST-NeRF: Using temporal aggregation for Zero-Shot Temporal NeRFs](https://mlsanity.com/paper/2311.18491/) (2023)
- [Gaussians on Fire: High-Frequency Reconstruction of Flames](https://mlsanity.com/paper/2511.22459/) (2025)
- [Complet4R: Geometric Complete 4D Reconstruction](https://mlsanity.com/paper/2603.27300/) (2026)
- [Rethinking Dense Optical Flow without Test-Time Scaling](https://mlsanity.com/paper/2605.08000/) (2026)

## Abstract (arXiv)

This paper proposes a new approach for monocular dense 3D reconstruction of a complex dynamic scene from two perspective frames. By applying superpixel over-segmentation to the image, we model a generically dynamic (hence non-rigid) scene with a piecewise planar and rigid approximation. In this way, we reduce the dynamic reconstruction problem to a "3D jigsaw puzzle" problem which takes pieces from an unorganized "soup of superpixels". We show that our method provides an effective solution to the inherent relative scale ambiguity in structure-from-motion. Since our method does not assume a template prior, or per-object segmentation, or knowledge about the rigidity of the dynamic scene, it is applicable to a wide range of scenarios. Extensive experiments on both synthetic and real monocular sequences demonstrate the superiority of our method compared with the state-of-the-art methods.