mlsanityData to 14 Oct 2026

VisionNVS: Self-Supervised Inpainting for Novel View Synthesis under the Virtual-Shift Paradigm

arXiv 2603.17382 ↗First posted 18 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

A fundamental bottleneck in Novel View Synthesis (NVS) for autonomous driving is the inherent supervision gap on novel trajectories: models are tasked with synthesizing unseen views during inference, yet lack ground truth images for these shifted poses during training. In this paper, we propose VisionNVS, a camera-only framework that fundamentally reformulates view synthesis from an ill-posed extrapolation problem into a self-supervised inpainting task. By introducing a ``Virtual-Shift'' strategy, we use monocular depth proxies to simulate occlusion patterns and map them onto the original view. This paradigm shift allows the use of raw, recorded images as pixel-perfect supervision, effectively eliminating the domain gap inherent in previous approaches. Furthermore, we address spatial consistency through a Pseudo-3D Seam Synthesis strategy, which integrates visual data from adjacent cameras during training to explicitly model real-world photometric discrepancies and calibration errors. Experiments demonstrate that VisionNVS achieves superior geometric fidelity and visual quality compared to LiDAR-dependent baselines, offering a robust solution for scalable driving simulation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

21 of 27 references.

References · 21

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  4. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2024Flow Matching Guide and Codecited by 301
  7. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  8. 2023EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervisioncited by 217
  9. 2024OmniRe: Omni Urban Scene Reconstructioncited by 161
  10. 2023Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Renderingcited by 143
  11. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  12. 20243D Gaussian as a New Era: A Surveycited by 104
9 more
  1. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
  2. 2024DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Inputcited by 53
  3. 2024FreeVS: Generative View Synthesis on Free Driving Trajectorycited by 53
  4. 2025DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generationcited by 40
  5. 2025RemoteSAM: Towards Segment Anything for Earth Observationcited by 39
  6. 2025RemoteReasoner: Towards Unifying Geospatial Reasoning Workflowcited by 28
  7. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  8. 2024EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splattingcited by 15
  9. 2024Toy-GS: Assembling Local Gaussians for Precisely Rendering Large-Scale Free Camera Trajectoriescited by 3

Built on by

Not cited yet.