mlsanityData to 7 Oct 2026

Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation

arXiv 2403.19319 ↗First posted 28 Mar 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

We present Mesh2NeRF, an approach to derive ground-truth radiance fields from textured meshes for 3D generation tasks. Many 3D generative approaches represent 3D scenes as radiance fields for training. Their ground-truth radiance fields are usually fitted from multi-view renderings from a large-scale synthetic 3D dataset, which often results in artifacts due to occlusions or under-fitting issues. In Mesh2NeRF, we propose an analytic solution to directly obtain ground-truth radiance fields from 3D meshes, characterizing the density field with an occupancy function featuring a defined surface thickness, and determining view-dependent color through a reflection function considering both the mesh and environment lighting. Mesh2NeRF extracts accurate radiance fields which provides direct supervision for training generative NeRFs and single scene representation. We validate the effectiveness of Mesh2NeRF across various tasks, achieving a noteworthy 3.12dB improvement in PSNR for view synthesis in single scene representation on the ABO dataset, a 0.69 PSNR enhancement in the single-view conditional generation of ShapeNet Cars, and notably improved mesh extraction from NeRF in the unconditional generation of Objaverse Mugs.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 62 of its 70 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

62 of 70 references.

References · 62

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  10. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
38 more
  1. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  5. 2022Diffusion Models in Vision: A Surveycited by 1,008
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  8. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  9. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  10. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  11. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  13. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  14. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  15. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  16. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  17. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  18. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  19. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  20. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  21. 2017Neural 3D Mesh Renderercited by 437
  22. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  23. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  24. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  25. 2021NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Imagescited by 336
  26. 2021Neural RGB-D Surface Reconstructioncited by 312
  27. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  28. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  29. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  30. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  31. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  32. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  33. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  34. 2023BakedSDF: Meshing Neural SDFs for Real-Time View Synthesiscited by 220
  35. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  36. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  37. 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 191
  38. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusioncited by 36
  2. 2025LinPrim: Linear Primitives for Differentiable Volumetric Renderingcited by 21
  3. 2025PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priorscited by 4
  4. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  5. 2026FlowObject: Flow Steering for Bridging Generative Priors and Reconstruction Fidelitycited by 1