# 3D Moments from Near-Duplicate Photos

arXiv: https://arxiv.org/abs/2205.06255 · HTML: https://mlsanity.com/paper/2205.06255/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 62 references that are in the map.
- First posted on arXiv 12 May 2022.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera](https://mlsanity.com/paper/2004.01294/) (2020)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)

## Built on by (most cited first, 15 of 15)

- [Generative Image Dynamics](https://mlsanity.com/paper/2309.07906/) (2023)
- [Removing Objects From Neural Radiance Fields](https://mlsanity.com/paper/2212.11966/) (2022)
- [Flow supervision for Deformable NeRF](https://mlsanity.com/paper/2303.16333/) (2023)
- [3D Cinemagraphy from a Single Image](https://mlsanity.com/paper/2303.05724/) (2023)
- [Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image](https://mlsanity.com/paper/2308.10257/) (2023)
- [RI3D: Few-Shot Gaussian Splatting With Repair and Inpainting Diffusion Priors](https://mlsanity.com/paper/2503.10860/) (2025)
- [Learning 3D Photography Videos via Self-supervised Diffusion on Single Images](https://mlsanity.com/paper/2302.10781/) (2023)
- [SpatialMe: Stereo Video Conversion Using Depth-Warping and Blend-Inpainting](https://mlsanity.com/paper/2412.11512/) (2024)
- [3D Video Loops from Asynchronous Input](https://mlsanity.com/paper/2303.05312/) (2023)
- [Optimizing 4D Gaussians for Dynamic Scene Video from Single Landscape Images](https://mlsanity.com/paper/2504.05458/) (2025)
- [ReShader: View-Dependent Highlights for Single Image View-Synthesis](https://mlsanity.com/paper/2309.10689/) (2023)
- [MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising](https://mlsanity.com/paper/2312.10899/) (2023)
- [ROSGS: Relightable Outdoor Scenes With Gaussian Splatting](https://mlsanity.com/paper/2509.11275/) (2025)
- [Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representation](https://mlsanity.com/paper/2407.21450/) (2024)
- [Eulerian Motion Reconstruction for Water Scenery](https://mlsanity.com/paper/2609.38622/) (2026)

## Abstract (arXiv)

We introduce 3D Moments, a new computational photography effect. As input we take a pair of near-duplicate photos, i.e., photos of moving subjects from similar viewpoints, common in people's photo collections. As output, we produce a video that smoothly interpolates the scene motion from the first photo to the second, while also producing camera motion with parallax that gives a heightened sense of 3D. To achieve this effect, we represent the scene as a pair of feature-based layered depth images augmented with scene flow. This representation enables motion interpolation along with independent control of the camera viewpoint. Our system produces photorealistic space-time videos with motion parallax and scene dynamics, while plausibly recovering regions occluded in the original views. We conduct extensive experiments demonstrating superior performance over baselines on public datasets and in-the-wild photos. Project page: https://3d-moments.github.io/