# Unsupervised Discovery and Composition of Object Light Fields

arXiv: https://arxiv.org/abs/2205.03923 · HTML: https://mlsanity.com/paper/2205.03923/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 59 references that are in the map.
- First posted on arXiv 8 May 2022.

## Builds on (most cited first, 30 of 40)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Conditional Object-Centric Learning from Video](https://mlsanity.com/paper/2111.12594/) (2021)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)

## Built on by (most cited first, 28 of 28)

- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Object-Centric Slot Diffusion](https://mlsanity.com/paper/2303.10834/) (2023)
- [SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models](https://mlsanity.com/paper/2305.11281/) (2023)
- [NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes](https://mlsanity.com/paper/2209.08776/) (2022)
- [Disentangled 3D Scene Generation with Layout Learning](https://mlsanity.com/paper/2402.16936/) (2024)
- [Rotating Features for Object Discovery](https://mlsanity.com/paper/2306.00600/) (2023)
- [Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation](https://mlsanity.com/paper/2210.00489/) (2022)
- [Neural Groundplans: Persistent Neural Scene Representations from a Single Image](https://mlsanity.com/paper/2207.11232/) (2022)
- [Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis](https://mlsanity.com/paper/2308.02840/) (2023)
- [Semantic Attention Flow Fields for Monocular Dynamic Scene Decomposition](https://mlsanity.com/paper/2303.01526/) (2023)
- [AutoRecon: Automated 3D Object Discovery and Reconstruction](https://mlsanity.com/paper/2305.08810/) (2023)
- [Unsupervised Discovery of Object-Centric Neural Fields](https://mlsanity.com/paper/2402.07376/) (2024)
- [SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields](https://mlsanity.com/paper/2408.06697/) (2024)
- [Semantic-aware Occlusion Filtering Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2303.03966/) (2023)
- [Multi-Object Manipulation via Object-Centric Neural Scattering Functions](https://mlsanity.com/paper/2306.08748/) (2023)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [Progressively-connected Light Field Network for Efficient View Synthesis](https://mlsanity.com/paper/2207.04465/) (2022)
- [Neural Radiance Field Codebooks](https://mlsanity.com/paper/2301.04101/) (2023)
- [Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations](https://mlsanity.com/paper/2310.17880/) (2023)
- [Binding Dynamics in Rotating Features](https://mlsanity.com/paper/2402.05627/) (2024)
- [Leveraging Image Augmentation for Object Manipulation: Towards Interpretable Controllability in Object-Centric Learning](https://mlsanity.com/paper/2310.08929/) (2023)
- [Slot-guided Volumetric Object Radiance Fields](https://mlsanity.com/paper/2401.02241/) (2024)
- [A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations](https://mlsanity.com/paper/2603.14022/) (2026)
- [Scene-Agnostic Object-Centric Representation Learning for 3D Gaussian Splatting](https://mlsanity.com/paper/2604.09045/) (2026)
- [3D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learning](https://mlsanity.com/paper/2606.19451/) (2026)
- [Variational Inference for Scalable 3D Object-centric Learning](https://mlsanity.com/paper/2309.14010/) (2023)
- [HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition](https://mlsanity.com/paper/2607.08249/) (2026)
- [ParticleSplat: Self-supervised Object-centric Latent Particle Splatting](https://mlsanity.com/paper/2609.19463/) (2026)

## Abstract (arXiv)

Neural scene representations, both continuous and discrete, have recently emerged as a powerful new paradigm for 3D scene understanding. Recent efforts have tackled unsupervised discovery of object-centric neural scene representations. However, the high cost of ray-marching, exacerbated by the fact that each object representation has to be ray-marched separately, leads to insufficiently sampled radiance fields and thus, noisy renderings, poor framerates, and high memory and time complexity during training and rendering. Here, we propose to represent objects in an object-centric, compositional scene representation as light fields. We propose a novel light field compositor module that enables reconstructing the global light field from a set of object-centric light fields. Dubbed Compositional Object Light Fields (COLF), our method enables unsupervised learning of object-centric neural scene representations, state-of-the-art reconstruction and novel view synthesis performance on standard datasets, and rendering and training speeds at orders of magnitude faster than existing 3D approaches.