# 3D Video Loops from Asynchronous Input

arXiv: https://arxiv.org/abs/2303.05312 · HTML: https://mlsanity.com/paper/2303.05312/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 60 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields](https://mlsanity.com/paper/2106.13228/) (2021)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [DeepStereo: Learning to Predict New Views from the World's Imagery](https://mlsanity.com/paper/1506.06825/) (2015)
- [Neural Volumes: Learning Dynamic Renderable Volumes from Images](https://mlsanity.com/paper/1906.07751/) (2019)
- [Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera](https://mlsanity.com/paper/2004.01294/) (2020)
- [Stable View Synthesis](https://mlsanity.com/paper/2011.07233/) (2020)
- [Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time](https://mlsanity.com/paper/2202.08614/) (2022)
- [T"oRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis](https://mlsanity.com/paper/2109.15271/) (2021)
- [4D Visualization of Dynamic Events from Unconstrained Multi-View Videos](https://mlsanity.com/paper/2005.13532/) (2020)
- [Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera](https://mlsanity.com/paper/2206.15258/) (2022)
- [Animating Pictures with Eulerian Motion Fields](https://mlsanity.com/paper/2011.15128/) (2020)

## Built on by (most cited first, 5 of 5)

- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Modeling Ambient Scene Dynamics for Free-view Synthesis](https://mlsanity.com/paper/2406.09395/) (2024)
- [LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field](https://mlsanity.com/paper/2404.08966/) (2024)
- [Eulerian Motion Reconstruction for Water Scenery](https://mlsanity.com/paper/2609.38622/) (2026)
- [Make It Flow: Animating Water in 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2610.13860/) (2026)

## Abstract (arXiv)

Looping videos are short video clips that can be looped endlessly without visible seams or artifacts. They provide a very attractive way to capture the dynamism of natural scenes. Existing methods have been mostly limited to 2D representations. In this paper, we take a step forward and propose a practical solution that enables an immersive experience on dynamic 3D looping scenes. The key challenge is to consider the per-view looping conditions from asynchronous input while maintaining view consistency for the 3D representation. We propose a novel sparse 3D video representation, namely Multi-Tile Video (MTV), which not only provides a view-consistent prior, but also greatly reduces memory usage, making the optimization of a 4D volume tractable. Then, we introduce a two-stage pipeline to construct the 3D looping MTV from completely asynchronous multi-view videos with no time overlap. A novel looping loss based on video temporal retargeting algorithms is adopted during the optimization to loop the 3D scene. Experiments of our framework have shown promise in successfully generating and rendering photorealistic 3D looping videos in real time even on mobile devices. The code, dataset, and live demos are available in https://limacv.github.io/VideoLoop3D_web/.