# MAIR++: Improving Multi-view Attention Inverse Rendering with Implicit Lighting Representation arXiv: https://arxiv.org/abs/2408.06707 ยท HTML: https://mlsanity.com/paper/2408.06707/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 59 of its 81 references that are in the map. - First posted on arXiv 13 Aug 2024. ## Builds on (most cited first, 30 of 59) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014) - [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021) - [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021) - [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021) - [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020) - [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020) - [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021) - [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021) - [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021) - [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023) - [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019) - [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022) - [TensoIR: Tensorial Inverse Rendering](https://mlsanity.com/paper/2304.12461/) (2023) - [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017) - [Editable Free-viewpoint Video Using a Layered Neural Representation](https://mlsanity.com/paper/2104.14786/) (2021) - [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023) - [Hallucinated Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2111.15246/) (2021) - [Single-Image SVBRDF Capture with a Rendering-Aware Deep Network](https://mlsanity.com/paper/1810.09718/) (2018) - [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022) - [Object-Centric Neural Scene Rendering](https://mlsanity.com/paper/2012.08503/) (2020) - [Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes](https://mlsanity.com/paper/2304.03266/) (2023) - [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018) - [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018) - [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019) - [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021) ## Built on by (most cited first, 3 of 3) - [LuxRemix: Lighting Decomposition and Remixing for Indoor Scenes](https://mlsanity.com/paper/2601.15283/) (2026) - [MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds](https://mlsanity.com/paper/2512.21003/) (2025) - [Channel-wise Noise Scheduled Diffusion for Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2503.09993/) (2025) ## Abstract (arXiv) In this paper, we propose a scene-level inverse rendering framework that uses multi-view images to decompose the scene into geometry, SVBRDF, and 3D spatially-varying lighting. While multi-view images have been widely used for object-level inverse rendering, scene-level inverse rendering has primarily been studied using single-view images due to the lack of a dataset containing high dynamic range multi-view images with ground-truth geometry, material, and spatially-varying lighting. To improve the quality of scene-level inverse rendering, a novel framework called Multi-view Attention Inverse Rendering (MAIR) was recently introduced. MAIR performs scene-level multi-view inverse rendering by expanding the OpenRooms dataset, designing efficient pipelines to handle multi-view images, and splitting spatially-varying lighting. Although MAIR showed impressive results, its lighting representation is fixed to spherical Gaussians, which limits its ability to render images realistically. Consequently, MAIR cannot be directly used in applications such as material editing. Moreover, its multi-view aggregation networks have difficulties extracting rich features because they only focus on the mean and variance between multi-view features. In this paper, we propose its extended version, called MAIR++. MAIR++ addresses the aforementioned limitations by introducing an implicit lighting representation that accurately captures the lighting conditions of an image while facilitating realistic rendering. Furthermore, we design a directional attention-based multi-view aggregation network to infer more intricate relationships between views. Experimental results show that MAIR++ not only achieves better performance than MAIR and single-view-based methods, but also displays robust performance on unseen real-world scenes.