# Appearance Editing with Free-viewpoint Neural Rendering

arXiv: https://arxiv.org/abs/2110.07674 · HTML: https://mlsanity.com/paper/2110.07674/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)
- [Stereo Magnification: Learning View Synthesis using Multiplane Images](https://mlsanity.com/paper/1805.09817/) (2018)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [Stable View Synthesis](https://mlsanity.com/paper/2011.07233/) (2020)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Inverse Path Tracing for Joint Material and Lighting Estimation](https://mlsanity.com/paper/1903.07145/) (2019)
- [A Neural Rendering Framework for Free-Viewpoint Relighting](https://mlsanity.com/paper/1911.11530/) (2019)
- [Transformable Bottleneck Networks](https://mlsanity.com/paper/1904.06458/) (2019)
- [Geometry-aware Deep Network for Single-Image Novel View Synthesis](https://mlsanity.com/paper/1804.06008/) (2018)

## Built on by (most cited first, 1 of 1)

- [Environment Maps Editing using Inverse Rendering and Adversarial Implicit Functions](https://mlsanity.com/paper/2410.18622/) (2024)

## Abstract (arXiv)

We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi-view images captured under known environment illumination. Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance. Our approach explicitly disentangles the appearance and learns a lighting representation that is independent of it. Specifically, we independently estimate the BRDF and use it to learn a lighting-only representation of the scene. Such disentanglement allows our approach to generalize to arbitrary changes in appearance while performing view synthesis. We show results of editing the appearance of a real scene, demonstrating that our approach produces plausible appearance editing. The performance of our view synthesis approach is demonstrated to be at par with state-of-the-art approaches on both real and synthetic data.