# FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling

arXiv: https://arxiv.org/abs/2104.08418 · HTML: https://mlsanity.com/paper/2104.08418/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 30 of 33)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild](https://mlsanity.com/paper/1911.11130/) (2019)
- [Escaping Plato's Cave: 3D Shape From Adversarial Rendering](https://mlsanity.com/paper/1811.11606/) (2018)
- [Optimizing the Latent Space of Generative Networks](https://mlsanity.com/paper/1707.05776/) (2017)
- [Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations](https://mlsanity.com/paper/2012.09988/) (2020)
- [3D Shape Induction from 2D Views of Multiple Objects](https://mlsanity.com/paper/1612.05872/) (2016)
- [Semantic Amodal Segmentation](https://mlsanity.com/paper/1509.01329/) (2015)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)
- [STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering](https://mlsanity.com/paper/2101.01602/) (2020)
- [Unsupervised Learning of 3D Object Categories from Videos in the Wild](https://mlsanity.com/paper/2103.16552/) (2021)
- [Shelf-Supervised Mesh Prediction in the Wild](https://mlsanity.com/paper/2102.06195/) (2021)
- [Category-Specific Object Reconstruction from a Single Image](https://mlsanity.com/paper/1411.6069/) (2014)
- [SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization](https://mlsanity.com/paper/1912.05027/) (2019)
- [Learning 3D Object Categories by Looking Around Them](https://mlsanity.com/paper/1705.03951/) (2017)

## Built on by (most cited first, 30 of 47)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations](https://mlsanity.com/paper/2209.03494/) (2022)
- [NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)](https://mlsanity.com/paper/2210.00379/) (2022)
- [NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing](https://mlsanity.com/paper/2207.11911/) (2022)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [LOLNeRF: Learn from One Look](https://mlsanity.com/paper/2111.09996/) (2021)
- [Learning to Stylize Novel Views](https://mlsanity.com/paper/2105.13509/) (2021)
- [AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2207.10312/) (2022)
- [CoNeRF: Controllable Neural Radiance Fields](https://mlsanity.com/paper/2112.01983/) (2021)
- [Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion](https://mlsanity.com/paper/2306.04633/) (2023)
- [DOVE: Learning Deformable 3D Objects by Watching Videos](https://mlsanity.com/paper/2107.10844/) (2021)
- [AutoRF: Learning 3D Object Radiance Fields from Single View Observations](https://mlsanity.com/paper/2204.03593/) (2022)
- [NeuralDiff: Segmenting 3D objects that move in egocentric videos](https://mlsanity.com/paper/2110.09936/) (2021)
- [Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation](https://mlsanity.com/paper/2210.10108/) (2022)
- [Self-supervised Neural Articulated Shape and Appearance Models](https://mlsanity.com/paper/2205.08525/) (2022)
- [EPIC Fields: Marrying 3D Geometry and Video Understanding](https://mlsanity.com/paper/2306.08731/) (2023)
- [Reconstructing Animatable Categories from Videos](https://mlsanity.com/paper/2305.06351/) (2023)
- [NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping](https://mlsanity.com/paper/2303.10709/) (2023)
- [NeROIC: Neural Rendering of Objects from Online Image Collections](https://mlsanity.com/paper/2201.02533/) (2022)
- [Mitigating Motion Blur in Neural Radiance Fields with Events and Frames](https://mlsanity.com/paper/2403.19780/) (2024)
- [EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Points](https://mlsanity.com/paper/2212.04247/) (2022)
- [MEGANE: Morphable Eyeglass and Avatar Network](https://mlsanity.com/paper/2302.04868/) (2023)
- [CG-NeRF: Conditional Generative Neural Radiance Fields](https://mlsanity.com/paper/2112.03517/) (2021)
- [DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting](https://mlsanity.com/paper/2503.24210/) (2025)
- [Explicit Neural Surfaces: Learning Continuous Geometry With Deformation Fields](https://mlsanity.com/paper/2306.02956/) (2023)
- [CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Rendering](https://mlsanity.com/paper/2311.15510/) (2023)
- [Semantic-aware Occlusion Filtering Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2303.03966/) (2023)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects](https://mlsanity.com/paper/2209.04183/) (2022)

## Abstract (arXiv)

We investigate the use of Neural Radiance Fields (NeRF) to learn high quality 3D object category models from collections of input images. In contrast to previous work, we are able to do this whilst simultaneously separating foreground objects from their varying backgrounds. We achieve this via a 2-component NeRF model, FiG-NeRF, that prefers explanation of the scene as a geometrically constant background and a deformable foreground that represents the object category. We show that this method can learn accurate 3D object category models using only photometric supervision and casually captured images of the objects. Additionally, our 2-part decomposition allows the model to perform accurate and crisp amodal segmentation. We quantitatively evaluate our method with view synthesis and image fidelity metrics, using synthetic, lab-captured, and in-the-wild data. Our results demonstrate convincing 3D object category modelling that exceed the performance of existing methods.