mlsanityData to 14 Oct 2026

DVSM: Decoder-only View Synthesis Model Done Right

arXiv 2605.29891 ↗First posted 28 May 2026cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent Large View Synthesis Models (LVSMs) advocate an encoder-decoder architecture that separates reconstruction and rendering into distinct networks. We re-examine this design. Through controlled experiments, we show that a decoder-only architecture, which represents scenes implicitly as a KV-cache, outperforms encoder-decoder variants while using fewer parameters at identical rendering complexity. Further analysis shows that sharing weights between the color-input reconstruction network and the camera-only rendering network better aligns their features at the same viewpoint, facilitating image synthesis. Building on this finding, our model, dubbed DVSM, further incorporates foundation model priors and stage-wise patch sizing for an improved efficiency-quality tradeoff. Our results establish a new state of the art for novel-view synthesis across multiple benchmarks, in some cases even outperforming per-scene-optimized 3DGS under dense input views.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 90 of its 99 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

90 of 99 references.

References · 90

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  9. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  10. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  12. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
38 more
  1. 2025DINOv3cited by 1,414
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  3. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  4. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  5. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  6. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  7. 2022Token Merging: Your ViT But Fastercited by 1,021
  8. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  9. 2020On Layer Normalization in the Transformer Architecturecited by 922
  10. 2023Vision Transformers Need Registerscited by 871
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  12. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  13. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  14. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  15. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  16. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  17. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  18. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  19. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  20. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  21. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  22. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  23. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
  24. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  25. 2022Efficiently Scaling Transformer Inferencecited by 539
  26. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 417
  27. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  28. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  29. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  30. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  31. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  32. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  33. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 286
  34. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  35. 2020Query-Key Normalization for Transformerscited by 280
  36. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 248
  37. 2019Group-wise Correlation Stereo Networkcited by 235
  38. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026AIMS: Anchor-Integrated Multi-View Synthesis for Scalable Novel View Renderingcited by 0
  2. 2026GLoD-LVSM: Learning Geometry-Grounded Continuous-LoD Scene Representations for Large View Synthesis Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stabilizing Camera-Controlled Novel View Synthesis at Inference Time
  2. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observations
  3. 2024Novel View Synthesis with Pixel-Space Diffusion Models
  4. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views
  5. 2025Scaling Transformer-Based Novel View Synthesis Models with Token Disentanglement and Synthetic Data
  6. 2026OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis
  7. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model
  8. 2026D'ej`a View: Looping Transformers for Multi-View 3D Reconstruction