mlsanityData to 7 Oct 2026

Unsupervised Discovery of Object Radiance Fields

arXiv 2107.07905 ↗First posted 16 Jul 2021cs.CV cs.AIcited by 100

Not on the map. Why?

Abstract (arXiv)

We study the problem of inferring an object-centric scene representation from a single image, aiming to derive a representation that explains the image formation process, captures the scene's 3D nature, and is learned without supervision. Most existing methods on scene decomposition lack one or more of these characteristics, due to the fundamental challenge in integrating the complex 3D-to-2D image formation process into powerful inference schemes like deep networks. In this paper, we propose unsupervised discovery of Object Radiance Fields (uORF), integrating recent progresses in neural 3D scene representations and rendering with deep inference networks for unsupervised 3D scene decomposition. Trained on multi-view RGB images without annotations, uORF learns to decompose complex scenes with diverse, textured background from a single image. We show that uORF enables novel tasks, such as scene segmentation and editing in 3D, and it performs well on these tasks and on novel view synthesis on three datasets.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Builds on 48 of its 74 references that are in the map. First posted on arXiv 16 Jul 2021.

Builds on

48 of 74 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  10. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
36 more
  1. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  2. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  3. 2020Neural Sparse Voxel Fieldscited by 959
  4. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  5. 2020Object-Centric Learning with Slot Attentioncited by 865
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  10. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  11. 2018Which Training Methods for GANs do actually Converge?cited by 672
  12. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  13. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  14. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
  15. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  16. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  17. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  18. 2020State of the Art on Neural Renderingcited by 276
  19. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  20. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  21. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  22. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  23. 2020DeRF: Decomposed Radiance Fieldscited by 163
  24. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  25. 2017Neural Expectation Maximizationcited by 123
  26. 2020Differentiable Rendering: A Surveycited by 123
  27. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  28. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  29. 2020Object-Centric Neural Scene Renderingcited by 90
  30. 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
  31. 2015Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposalscited by 69
  32. 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
  33. 2020Toward unsupervised, multi-object discovery in large-scale image collectionscited by 65
  34. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  35. 2021Unsupervised Layered Image Decomposition into Object Prototypescited by 50
  36. 20183D-Aware Scene Manipulation via Inverse Graphicscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 25; 2023: 36; 2024: 23; 2025: 4; 2026: 8.

100 papers.

Sort:

By area:

  1. 2021Neural Fields in Visual Computing and Beyondcited by 539
  2. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  3. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
  4. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  5. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  6. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  7. 2023Segment Any 3D Gaussianscited by 158
  8. 2023SUDS: Scalable Urban Dynamic Scenescited by 151
  9. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  10. 2022Object Scene Representation Transformercited by 90
  11. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  12. 2022Object-Compositional Neural Implicit Surfacescited by 74
38 more
  1. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  2. 2021CoNeRF: Controllable Neural Radiance Fieldscited by 65
  3. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  4. 2023Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusioncited by 62
  5. 2022Deformable Sprites for Unsupervised Video Decompositioncited by 61
  6. 2022Neural Volumetric Object Selectioncited by 61
  7. 2023Segment Anything in 3D with Radiance Fieldscited by 61
  8. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  9. 2024FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimallycited by 57
  10. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  11. 2023EPIC Fields: Marrying 3D Geometry and Video Understandingcited by 49
  12. 2022Reinforcement Learning with Neural Radiance Fieldscited by 43
  13. 2023Instance Neural Radiance Fieldcited by 39
  14. 2022LaTeRF: Label and Text Driven Object Radiance Fieldscited by 38
  15. 2022Fast Learning Radiance Fields by Shooting Much Fewer Rayscited by 38
  16. 2023ObjectSDF++: Improved Object-Compositional Neural Implicit Surfacescited by 38
  17. 2022PartAfford: Part-level Affordance Discovery from 3D Objectscited by 35
  18. 2022BlobGAN: Spatially Disentangled Scene Representationscited by 35
  19. 2023OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fieldscited by 34
  20. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  21. 2023VQ3D: Learning a 3D-Aware Generative Model on ImageNetcited by 32
  22. 2022EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Pointscited by 30
  23. 2022Unsupervised Discovery and Composition of Object Light Fieldscited by 28
  24. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  25. 2022Compositional Scene Representation Learning via Reconstruction: A Surveycited by 27
  26. 2023GeoVLN: Learning Geometry-Enhanced Visual Representation with Slot Attention for Vision-and-Language Navigationcited by 24
  27. 2022Unsupervised Multi-View Object Segmentation Using Radiance Field Propagationcited by 22
  28. 2023Towards Unsupervised Object Detection From LiDAR Point Cloudscited by 22
  29. 2023Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Modelcited by 20
  30. 2023SANeRF-HQ: Segment Anything for NeRF in High Qualitycited by 20
  31. 2021LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networkscited by 19
  32. 2022PaletteNeRF: Palette-based Color Editing for NeRFscited by 18
  33. 2023Inferring Hybrid Neural Fluid Fields from Videoscited by 18
  34. 2022Neural Groundplans: Persistent Neural Scene Representations from a Single Imagecited by 17
  35. 2023NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objectscited by 17
  36. 2023DORSal: Diffusion for Object-centric Representations of Scenes et alcited by 16
  37. 2024NeRFs in Robotics: A Surveycited by 16
  38. 2023Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Compositioncited by 15

50 most cited of 100 All 100 (JSON)