mlsanityData to 7 Oct 2026

SegNeRF: 3D Part Segmentation with Neural Radiance Fields

arXiv 2211.11215 ↗First posted 21 Nov 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent advances in Neural Radiance Fields (NeRF) boast impressive performances for generative tasks such as novel view synthesis and 3D reconstruction. Methods based on neural radiance fields are able to represent the 3D world implicitly by relying exclusively on posed images. Yet, they have seldom been explored in the realm of discriminative tasks such as 3D part segmentation. In this work, we attempt to bridge that gap by proposing SegNeRF: a neural field representation that integrates a semantic field along with the usual radiance field. SegNeRF inherits from previous works the ability to perform novel view synthesis and 3D reconstruction, and enables 3D part segmentation from a few images. Our extensive experiments on PartNet show that SegNeRF is capable of simultaneously predicting geometry, appearance, and semantic information from posed images, even for unseen objects. The predicted semantic fields allow SegNeRF to achieve an average mIoU of 30.30% for 2D novel view segmentation, and 37.46% for 3D part segmentation, boasting competitive performance against point-based methods by using only a few posed images. Additionally, SegNeRF is able to generate an explicit 3D model from a single image of an object taken in the wild, with its corresponding part segmentation.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 21 Nov 2022.

Builds on

33 of 42 references.

References · 33

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  10. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  11. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  12. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
21 more
  1. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  2. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  4. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  5. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  6. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  7. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  8. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  9. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  10. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  11. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  12. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  13. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  14. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  15. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  16. 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
  17. 2020A Closer Look at Local Aggregation Operators in Point Cloud Analysiscited by 87
  18. 20163D Shape Segmentation with Projective Convolutional Networkscited by 63
  19. 2018VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapescited by 37
  20. 2017Learned Multi-Patch Similaritycited by 24
  21. 2020Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearancecited by 9

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 4.

9 papers.

  1. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 133
  2. 2024N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fieldscited by 34
  3. 2023Interactive Segment Anything NeRF with Feature Imitationcited by 24
  4. 20232D-Guided 3D Gaussian Segmentationcited by 12
  5. 2024GradiSeg: Gradient-Guided Gaussian Segmentation with Enhanced 3D Boundary Precisioncited by 10
  6. 2023PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenescited by 9
  7. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  8. 2024Fast Sparse View Guided NeRF Update for Object Reconfigurationscited by 3
  9. 2023InceptionHuman: Controllable Prompt-to-NeRF for Photorealistic 3D Human Generationcited by 0