mlsanityData to 14 Oct 2026

Deep Learning on Object-centric 3D Neural Fields

arXiv 2312.13277 ↗First posted 20 Dec 2023cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

In recent years, Neural Fields (NFs) have emerged as an effective tool for encoding diverse continuous signals such as images, videos, audio, and 3D shapes. When applied to 3D data, NFs offer a solution to the fragmentation and limitations associated with prevalent discrete representations. However, given that NFs are essentially neural networks, it remains unclear whether and how they can be seamlessly integrated into deep learning pipelines for solving downstream tasks. This paper addresses this research problem and introduces nf2vec, a framework capable of generating a compact latent representation for an input NF in a single inference pass. We demonstrate that nf2vec effectively embeds 3D objects represented by the input NFs and showcase how the resulting embeddings can be employed in deep learning pipelines to successfully address various tasks, all while processing exclusively NFs. We test this framework on several NFs used to represent 3D surfaces, such as unsigned/signed distance and occupancy fields. Moreover, we demonstrate the effectiveness of our approach with more complex NFs that encompass both geometry and appearance of 3D objects such as neural radiance fields.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 49 of its 69 references that are in the map. First posted on arXiv 20 Dec 2023.

Builds on

49 of 69 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  11. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  12. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
37 more
  1. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  2. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  3. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  4. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  5. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  6. 2020Convolutional Occupancy Networkscited by 761
  7. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  8. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  9. 2021Neural Fields in Visual Computing and Beyondcited by 540
  10. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  11. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  12. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
  13. 2020Local Implicit Grid Representations for 3D Scenescited by 373
  14. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  15. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 291
  16. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  17. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  18. 2021COIN: COmpression with Implicit Neural representationscited by 233
  19. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  20. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  21. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  22. 2021Implicit Neural Representations for Image Compressioncited by 162
  23. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  24. 2020Predicting Neural Network Accuracy from Weightscited by 110
  25. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
  26. 2021Variational Relational Point Completion Networkcited by 104
  27. 2023NerfAcc: Efficient Sampling Accelerates NeRFscited by 90
  28. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  29. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  30. 2023Permutation Equivariant Neural Functionalscited by 70
  31. 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
  32. 2021Subdivision-Based Mesh Convolution Networkscited by 60
  33. 2021Implicit Neural Video Compressioncited by 52
  34. 2023Neural Functional Transformerscited by 52
  35. 2019Neural Network Branching for Neural Network Verificationcited by 45
  36. 2021Generating Adversarial Examples with Graph Neural Networkscited by 12
  37. 2023Neural Processing of Tri-Plane Hybrid Neural Fieldscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
  2. 2025CompGS++: Compressed Gaussian Splatting for Static and Dynamic Scene Representationcited by 4
  3. 2026Neural Field Tokenizations with Hierarchy and Spatial Locality Priorscited by 1
  4. 2025Weight Space Representation Learning on Diverse NeRF Architecturescited by 0
  5. 2026Weight-Space Mixture-of-Experts for Implicit Neural Representation Classificationcited by 0