One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generation
Not on the map. Why?
Abstract (arXiv)
Monocular novel-view synthesis has long required multi-view image pairs for supervision, limiting training to a narrow set of purpose-built datasets. We propose in-the-wild monocular pretraining: a frozen depth estimator lifts each source image into 3D and reprojects under sampled poses to yield pseudo-target views; masked losses restrict supervision to valid regions and an adversarial objective covers disoccluded areas. Scaled to 30 million uncurated images, this produces OVIE, requiring only a source image and target pose at inference. Prior work trains without multi-view data but needs a depth estimator at inference, or drops this dependency but requires multi-view training pairs; OVIE is the first to require neither. Without multi-view supervision, OVIE rivals in-domain baselines on RealEstate10K and surpasses all on DL3DV, producing the most multi-view-consistent trajectories of any geometry-free method; brief multi-view fine-tuning outperforms all geometry-free methods on their training domain. At 116 FPS, it is over 600x faster than the fastest baseline. Code and pretrained models are at https://github.com/kyutai-labs/ovie; video results are on the project page, https://kyutai.org/blog/2026-04-14-ovie/.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 82 of its 90 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
82 of 90 references.
References · 82
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2021Vision Transformers for Dense Predictioncited by 1,789
38 more
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2024Grounding Image Matching in 3D with MASt3Rcited by 960
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
- 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
- 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
- 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
- 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2021ImageNet-21K Pretraining for the Massescited by 607
- 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
- 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
- 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
- 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
- 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
- 2025Diffusion Transformers with Representation Autoencoderscited by 275
- 2020Single-View View Synthesis with Multiplane Imagescited by 263
- 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
- 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
- 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
- 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
- 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
50 most cited of 82 All 82 (JSON)
Built on by
Not cited yet.