# SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix

arXiv: https://arxiv.org/abs/2407.00367 · HTML: https://mlsanity.com/paper/2407.00367/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 61 references that are in the map.
- First posted on arXiv 29 Jun 2024.

## Builds on (most cited first, 30 of 48)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Photorealistic Video Generation with Diffusion Models](https://mlsanity.com/paper/2312.06662/) (2023)
- [Flexible Diffusion Modeling of Long Videos](https://mlsanity.com/paper/2205.11495/) (2022)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)

## Built on by (most cited first, 20 of 20)

- [StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videos](https://mlsanity.com/paper/2409.07447/) (2024)
- [M2SVid: End-to-End Inpainting and Refinement for Monocular-to-Stereo Video Conversion](https://mlsanity.com/paper/2505.16565/) (2025)
- [Restereo: Diffusion stereo video generation and restoration](https://mlsanity.com/paper/2506.06023/) (2025)
- [Eye2Eye: A Simple Approach for Monocular-to-Stereo Video Synthesis](https://mlsanity.com/paper/2505.00135/) (2025)
- [What Makes for a Good Stereoscopic Image?](https://mlsanity.com/paper/2412.21127/) (2024)
- [StereoWorld: Geometry-Aware Monocular-to-Stereo Video Generation](https://mlsanity.com/paper/2512.09363/) (2025)
- [Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding](https://mlsanity.com/paper/2512.14236/) (2025)
- [StereoPilot: Learning Unified and Efficient Stereo Conversion via Generative Priors](https://mlsanity.com/paper/2512.16915/) (2025)
- [Stereo World Model: Camera-Guided Stereo Video Generation](https://mlsanity.com/paper/2603.17375/) (2026)
- [AniGen: Unified S3 Fields for Animatable 3D Asset Generation](https://mlsanity.com/paper/2604.08746/) (2026)
- [StereoSpace: Depth-Free Synthesis of Stereo Geometry via End-to-End Diffusion in a Canonical Space](https://mlsanity.com/paper/2512.10959/) (2025)
- [Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Views](https://mlsanity.com/paper/2601.03362/) (2026)
- [Faithfulness to Refusal: A Causal Audit of Neuron Selectors](https://mlsanity.com/paper/2607.05355/) (2026)
- [Consistency-Preserving Diverse Video Generation](https://mlsanity.com/paper/2602.15287/) (2026)
- [DreamStereo: Towards Real-Time Stereo Inpainting for HD Videos](https://mlsanity.com/paper/2604.12270/) (2026)
- [S^2VG: 3D Stereoscopic and Spatial Video Generation via Denoising Frame Matrix](https://mlsanity.com/paper/2508.08048/) (2025)
- [StereoGenBench: A Synthetic Multi-Camera Benchmark for Stereo Generation under Controlled Baseline Regimes](https://mlsanity.com/paper/2605.23237/) (2026)
- [αDepth: Learning Single-Pass Soft Boundary Decomposition for Stereo Conversion](https://mlsanity.com/paper/2606.00386/) (2026)
- [Monocular Depth Estimation from a Single Image: Progress and Opportunities](https://mlsanity.com/paper/2609.01172/) (2026)
- [StereoEngine: Scaling Stereo Bimanual Robot Data without Stereo Cameras](https://mlsanity.com/paper/2610.12734/) (2026)

## Abstract (arXiv)

Video generation models have demonstrated great capabilities of producing impressive monocular videos, however, the generation of 3D stereoscopic video remains under-explored. We propose a pose-free and training-free approach for generating 3D stereoscopic videos using an off-the-shelf monocular video generation model. Our method warps a generated monocular video into camera views on stereoscopic baseline using estimated video depth, and employs a novel frame matrix video inpainting framework. The framework leverages the video generation model to inpaint frames observed from different timestamps and views. This effective approach generates consistent and semantically coherent stereoscopic videos without scene optimization or model fine-tuning. Moreover, we develop a disocclusion boundary re-injection scheme that further improves the quality of video inpainting by alleviating the negative effects propagated from disoccluded areas in the latent space. We validate the efficacy of our proposed method by conducting experiments on videos from various generative models, including Sora [4 ], Lumiere [2], WALT [8 ], and Zeroscope [ 42]. The experiments demonstrate that our method has a significant improvement over previous methods. The code will be released at urlhttps://daipengwa.github.io/SVGProjectPage.