# 3D Cinemagraphy from a Single Image

arXiv: https://arxiv.org/abs/2303.05724 · HTML: https://mlsanity.com/paper/2303.05724/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 76 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 30 of 59)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)

## Built on by (most cited first, 24 of 24)

- [Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling](https://mlsanity.com/paper/2401.15977/) (2024)
- [AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance](https://mlsanity.com/paper/2311.12886/) (2023)
- [4K4DGen: Panoramic 4D Generation at 4K Resolution](https://mlsanity.com/paper/2406.13527/) (2024)
- [Comp4D: LLM-Guided Compositional 4D Scene Generation](https://mlsanity.com/paper/2403.16993/) (2024)
- [3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation](https://mlsanity.com/paper/2403.09439/) (2024)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [Text-Guided Synthesis of Eulerian Cinemagraphs](https://mlsanity.com/paper/2307.03190/) (2023)
- [Fast View Synthesis of Casual Videos with Soup-of-Planes](https://mlsanity.com/paper/2312.02135/) (2023)
- [Explorative Inbetweening of Time and Space](https://mlsanity.com/paper/2403.14611/) (2024)
- [VividDream: Generating 3D Scene with Ambient Dynamics](https://mlsanity.com/paper/2405.20334/) (2024)
- [Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image](https://mlsanity.com/paper/2308.10257/) (2023)
- [Simulating the Real World: A Unified Survey of Multimodal Generative Models](https://mlsanity.com/paper/2503.04641/) (2025)
- [LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field](https://mlsanity.com/paper/2404.08966/) (2024)
- [Towards Robust Monocular Depth Estimation in Non-Lambertian Surfaces](https://mlsanity.com/paper/2408.06083/) (2024)
- [NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation](https://mlsanity.com/paper/2307.08695/) (2023)
- [S-DyRF: Reference-Based Stylized Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2403.06205/) (2024)
- [AnimateScene: Camera-controllable Animation in Any Scene](https://mlsanity.com/paper/2508.05982/) (2025)
- [Learning an Implicit Physics Model for Image-based Fluid Simulation](https://mlsanity.com/paper/2508.08254/) (2025)
- [DreamLoop: Controllable Cinemagraph Generation from a Single Photograph](https://mlsanity.com/paper/2601.02646/) (2026)
- [Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representation](https://mlsanity.com/paper/2407.21450/) (2024)
- [LivingWorld: Interactive 4D World Generation with Environmental Dynamics](https://mlsanity.com/paper/2604.01641/) (2026)
- [DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion](https://mlsanity.com/paper/2605.16807/) (2026)
- [Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions](https://mlsanity.com/paper/2607.28285/) (2026)
- [Make It Flow: Animating Water in 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2610.13860/) (2026)

## Abstract (arXiv)

We present 3D Cinemagraphy, a new technique that marries 2D image animation with 3D photography. Given a single still image as input, our goal is to generate a video that contains both visual content animation and camera motion. We empirically find that naively combining existing 2D image animation and 3D photography methods leads to obvious artifacts or inconsistent animation. Our key insight is that representing and animating the scene in 3D space offers a natural solution to this task. To this end, we first convert the input image into feature-based layered depth images using predicted depth values, followed by unprojecting them to a feature point cloud. To animate the scene, we perform motion estimation and lift the 2D motion into the 3D scene flow. Finally, to resolve the problem of hole emergence as points move forward, we propose to bidirectionally displace the point cloud as per the scene flow and synthesize novel views by separately projecting them into target image planes and blending the results. Extensive experiments demonstrate the effectiveness of our method. A user study is also conducted to validate the compelling rendering results of our method.