Adjustable Visual Appearance for Generalizable Novel View Synthesis
Not on the map. Why?
Abstract (arXiv)
We present a generalizable novel view synthesis method which enables modifying the visual appearance of an observed scene so rendered views match a target weather or lighting condition without any scene specific training or access to reference views at the target condition. Our method is based on a pretrained generalizable transformer architecture and is fine-tuned on synthetically generated scenes under different appearance conditions. This allows for rendering novel views in a consistent manner for 3D scenes that were not included in the training set, along with the ability to (i) modify their appearance to match the target condition and (ii) smoothly interpolate between different conditions. Experiments on real and synthetic scenes show that our method is able to generate 3D consistent renderings while making realistic appearance changes, including qualitative and quantitative comparisons. Please refer to our project page for video results: https://ava-nvs.github.io/
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 56 of its 64 references that are in the map. First posted on arXiv 2 Jun 2023.
Builds on
56 of 64 references.
References · 56
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Generative Adversarial Networkscited by 5,627
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
38 more
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
- 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
- 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
- 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
- 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
- 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 724
- 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
- 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
- 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
- 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
- 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
- 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
- 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
- 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
- 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
- 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 370
- 2017Toward Multimodal Image-to-Image Translationcited by 296
- 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
- 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
- 2018Arbitrary Style Transfer with Style-Attentional Networkscited by 198
- 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
- 2017Optimizing the Latent Space of Generative Networkscited by 157
- 2022Diffusion-based Image Translation using Disentangled Style and Content Representationcited by 149
- 2022ARF: Artistic Radiance Fieldscited by 147
- 2022StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learningcited by 140
- 2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
- 2022Generalizable Patch-Based Neural Renderingcited by 122
- 2022Is Attention All That NeRF Needs?cited by 113
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.