mlsanityData to 7 Oct 2026

One View Is Enough: In-the-Wild Monocular Pretraining for Novel View Generation

arXiv 2603.23488 ↗First posted 24 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Monocular novel-view synthesis has long required multi-view image pairs for supervision, limiting training to a narrow set of purpose-built datasets. We propose in-the-wild monocular pretraining: a frozen depth estimator lifts each source image into 3D and reprojects under sampled poses to yield pseudo-target views; masked losses restrict supervision to valid regions and an adversarial objective covers disoccluded areas. Scaled to 30 million uncurated images, this produces OVIE, requiring only a source image and target pose at inference. Prior work trains without multi-view data but needs a depth estimator at inference, or drops this dependency but requires multi-view training pairs; OVIE is the first to require neither. Without multi-view supervision, OVIE rivals in-domain baselines on RealEstate10K and surpasses all on DL3DV, producing the most multi-view-consistent trajectories of any geometry-free method; brief multi-view fine-tuning outperforms all geometry-free methods on their training domain. At 116 FPS, it is over 600x faster than the fastest baseline. Code and pretrained models are at https://github.com/kyutai-labs/ovie; video results are on the project page, https://kyutai.org/blog/2026-04-14-ovie/.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 82 of its 90 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

82 of 90 references.

References · 82

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  12. 2021Vision Transformers for Dense Predictioncited by 1,789
38 more
  1. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  2. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  3. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  6. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  10. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  11. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  13. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  14. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  15. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  16. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  17. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  18. 2021ImageNet-21K Pretraining for the Massescited by 607
  19. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  20. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  21. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  22. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  23. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  24. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  25. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  26. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  27. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  28. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  29. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  30. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  31. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  32. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  33. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  34. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  35. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  36. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  37. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  38. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212

50 most cited of 82 All 82 (JSON)

Built on by

Not cited yet.