mlsanityData to 7 Oct 2026

Free3D: Consistent Novel View Synthesis without 3D Representation

arXiv 2312.04551 ↗First posted 7 Dec 2023cs.CVcited by 72

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Free3D, a simple accurate method for monocular open-set novel view synthesis (NVS). Similar to Zero-1-to-3, we start from a pre-trained 2D image generator for generalization, and fine-tune it for NVS. Compared to other works that took a similar approach, we obtain significant improvements without resorting to an explicit 3D representation, which is slow and memory-consuming, and without training an additional network for 3D reconstruction. Our key contribution is to improve the way the target camera pose is encoded in the network, which we do by introducing a new ray conditioning normalization (RCN) layer. The latter injects pose information in the underlying 2D image generator by telling each pixel its viewing direction. We further improve multi-view consistency by using light-weight multi-view attention layers and by sharing generation noise between the different views. We train Free3D on the Objaverse dataset and demonstrate excellent generalization to new categories in new datasets, including OmniObject3D and GSO. The project page is available at https://chuanxiaz.com/free3d/.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 78 of its 95 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

78 of 95 references.

References · 78

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2023Segment Anythingcited by 8,158
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  12. 2017Neural Discrete Representation Learningcited by 4,650
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  8. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  9. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  13. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  14. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  15. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  16. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  17. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  18. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  19. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  20. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  21. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  22. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  23. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  24. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  25. 2016Pixel Recurrent Neural Networkscited by 814
  26. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  27. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  28. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  29. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  30. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  31. 2022MAGVIT: Masked Generative Video Transformercited by 341
  32. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  33. 2016A Learned Representation For Artistic Stylecited by 340
  34. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  35. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  36. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  37. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  38. 2022Novel View Synthesis with Diffusion Modelscited by 299

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 32; 2025: 24; 2026: 15.

72 papers.

Sort:

By area:

  1. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  2. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  3. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  4. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  5. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 178
  6. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  7. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  8. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  9. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
  10. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  11. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  12. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
38 more
  1. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 47
  2. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  3. 20243DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Viewscited by 37
  4. 2024DragAPart: Learning a Part-Level Motion Prior for Articulated Objectscited by 32
  5. 2024Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusioncited by 31
  6. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  7. 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundnesscited by 24
  8. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 19
  9. 20243D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generationcited by 19
  10. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representationcited by 17
  11. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observationscited by 16
  12. 2025AR-1-to-3: Single Image to Consistent 3D Object Generation via Next-View Predictioncited by 14
  13. 2025Edit360: 2D Image Edits to 3D Assets from Any Anglecited by 13
  14. 2024Flex3D: Feed-Forward 3D Generation with Flexible Reconstruction Model and Input View Curationcited by 12
  15. 2024GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generationcited by 12
  16. 20253DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Modelscited by 12
  17. 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
  18. 2024Drive-1-to-3: Enriching Diffusion Priors for Novel View Synthesis of Real Vehiclescited by 10
  19. 2024HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Modelcited by 9
  20. 2024DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priorscited by 9
  21. 2024MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenescited by 9
  22. 2025High-Fidelity Novel View Synthesis via Splatting-Guided Diffusioncited by 9
  23. 2025MVGBench: Comprehensive Benchmark for Multi-view Generation Modelscited by 9
  24. 2025MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attentioncited by 8
  25. 2024RGB2Point: 3D Point Cloud Generation from Single RGB Imagescited by 7
  26. 2024DiffAge3D: Diffusion-based 3D-aware Face Agingcited by 6
  27. 2026NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstructioncited by 5
  28. 2024Jointly Generating Multi-view Consistent PBR Textures using Collaborative Controlcited by 4
  29. 2025UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesiscited by 4
  30. 2026Syn4D: A Multiview Synthetic 4D Datasetcited by 4
  31. 2024LeanGaussian: Breaking Pixel or Point Cloud Correspondence in Modeling 3D Gaussianscited by 3
  32. 2024UniGS: Modeling Unitary 3D Gaussians for Novel View Synthesis from Sparse-view Imagescited by 3
  33. 2025WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Imagecited by 3
  34. 2025Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validitycited by 3
  35. 2026MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAEcited by 3
  36. 2024Pragmatist: Multiview Conditional Diffusion Models for High-Fidelity 3D Reconstruction from Unposed Sparse Viewscited by 2
  37. 2025MD-ProjTex: Texturing 3D Shapes with Multi-Diffusion Projectioncited by 2
  38. 2025Geometric Consistency Refinement for Single Image Novel View Synthesis via Test-Time Adaptation of Diffusion Modelscited by 2

50 most cited of 72 All 72 (JSON)