mlsanityData to 14 Oct 2026

LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias

arXiv 2410.17242 ↗First posted 22 Oct 2024cs.CV cs.GR cs.LGcited by 189

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose the Large View Synthesis Model (LVSM), a novel transformer-based approach for scalable and generalizable novel view synthesis from sparse-view inputs. We introduce two architectures: (1) an encoder-decoder LVSM, which encodes input image tokens into a fixed number of 1D latent tokens, functioning as a fully learned scene representation, and decodes novel-view images from them; and (2) a decoder-only LVSM, which directly maps input images to novel-view outputs, completely eliminating intermediate scene representations. Both models bypass the 3D inductive biases used in previous methods -- from 3D representations (e.g., NeRF, 3DGS) to network designs (e.g., epipolar projections, plane sweeps) -- addressing novel view synthesis with a fully data-driven approach. While the encoder-decoder model offers faster inference due to its independent latent representation, the decoder-only LVSM achieves superior quality, scalability, and zero-shot generalization, outperforming previous state-of-the-art methods by 1.5 to 3.5 dB PSNR. Comprehensive evaluations across multiple datasets demonstrate that both LVSM variants achieve state-of-the-art novel view synthesis quality. Notably, our models surpass all previous methods even with reduced computational resources (1-2 GPUs). Please see our website for more details: https://haian-jin.github.io/projects/LVSM/ .

Key facts. Cited by 189 papers in the map (data through 14 Oct 2026). Builds on 78 of its 97 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

78 of 97 references.

References · 78

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  12. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
38 more
  1. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  3. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  4. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  6. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  7. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  9. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  12. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  13. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  14. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  15. 2024Genie: Generative Interactive Environmentscited by 747
  16. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  17. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  18. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  19. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  20. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  21. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  22. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  23. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
  24. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  25. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  26. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
  27. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  28. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  29. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  30. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  31. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  32. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  33. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  34. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  35. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  36. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  37. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  38. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 72; 2026: 107.

189 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  2. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  3. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
  4. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 243
  5. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 182
  6. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  7. 2025Test-Time Training Done Rightcited by 130
  8. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 116
  9. 2025Cameras as Relative Positional Encodingcited by 105
  10. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 99
  11. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 93
  12. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 77
38 more
  1. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  2. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 49
  3. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  4. 2025Dens3R: A Foundation Model for 3D Geometry Predictioncited by 46
  5. 2025Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splattingcited by 36
  6. 2025UFM: A Simple Path towards Unified Dense Correspondence with Flowcited by 36
  7. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  8. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstructioncited by 31
  9. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 27
  10. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  11. 2026ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Trainingcited by 26
  12. 2025RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illuminationcited by 23
  13. 2024MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Datacited by 21
  14. 2025MoRE: 3D Visual Geometry Reconstruction Meets Mixture-of-Expertscited by 21
  15. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenescited by 20
  16. 2025Positional Encoding Fieldcited by 20
  17. 2025Sharp Monocular View Synthesis in Less Than a Secondcited by 20
  18. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priorscited by 19
  19. 2025iLRM: An Iterative Large 3D Reconstruction Modelcited by 19
  20. 2025ReSplat: Learning Recurrent Gaussian Splattingcited by 15
  21. 2025True Self-Supervised Novel View Synthesis is Transferablecited by 15
  22. 2026TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokenscited by 15
  23. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 14
  24. 20254D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Timecited by 14
  25. 2026RayRoPE: Projective Ray Positional Encoding for Multi-view Attentioncited by 14
  26. 2025LARM: A Large Articulated-Object Reconstruction Modelcited by 12
  27. 2026tttLRM: Test-Time Training for Long Context and Autoregressive 3D Reconstructioncited by 12
  28. 2026GlobalSplat: Efficient Feed-Forward 3D Gaussian Splatting via Global Scene Tokenscited by 12
  29. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 11
  30. 2025Off The Grid: Detection of Primitives for Feed-Forward 3D Gaussian Splattingcited by 11
  31. 2025SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Timecited by 11
  32. 2025Denoising Hamiltonian Network for Physical Reasoningcited by 10
  33. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  34. 2025InstaInpaint: Instant 3D-Scene Inpainting with Masked Large Reconstruction Modelcited by 10
  35. 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encodingcited by 10
  36. 2025PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesiscited by 10
  37. 2026SceneTok: A Compressed, Diffusable Token Space for 3D Scenescited by 10
  38. 2025HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformerscited by 9

50 most cited of 189 All 189 (JSON)