mlsanityData to 14 Oct 2026

Geometry Aware Field-to-field Transformations for 3D Semantic Segmentation

arXiv 2310.05133 ↗First posted 8 Oct 2023cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a novel approach to perform 3D semantic segmentation solely from 2D supervision by leveraging Neural Radiance Fields (NeRFs). By extracting features along a surface point cloud, we achieve a compact representation of the scene which is sample-efficient and conducive to 3D reasoning. Learning this feature space in an unsupervised manner via masked autoencoding enables few-shot segmentation. Our method is agnostic to the scene parameterization, working on scenes fit with any type of NeRF.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 63 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

47 of 63 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  10. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  12. 2020Point Transformercited by 1,470
35 more
  1. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  2. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  3. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  4. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
  5. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 724
  6. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  7. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
  8. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  9. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  10. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  11. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  12. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  13. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  14. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  15. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  16. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  17. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 398
  18. 2022Kubric: A scalable dataset generatorcited by 381
  19. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 314
  20. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  21. 2017Submanifold Sparse Convolutional Networkscited by 223
  22. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  23. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  24. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 167
  25. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  26. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  27. 2021HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputscited by 112
  28. 2023Grid-guided Neural Radiance Fields for Large Urban Scenescited by 100
  29. 2017Deep Projective 3D Semantic Segmentationcited by 81
  30. 2021InverseForm: A Loss Function for Structured Boundary-Aware Segmentationcited by 65
  31. 2018VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapescited by 37
  32. 2014Spatially-sparse convolutional neural networkscited by 33
  33. 2021Deep Learning Based 3D Segmentation: A Surveycited by 30
  34. 2022LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformerscited by 21
  35. 2022Masked Surfel Prediction for Self-Supervised Point Cloud Learningcited by 15

Built on by

Not cited yet.