# Intrinsic Image Decomposition Using Point Cloud Representation

arXiv: https://arxiv.org/abs/2307.10924 · HTML: https://mlsanity.com/paper/2307.10924/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [Learning Intrinsic Image Decomposition from Watching the World](https://mlsanity.com/paper/1804.00582/) (2018)
- [OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets](https://mlsanity.com/paper/2007.12868/) (2020)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [Unsupervised Learning for Intrinsic Image Decomposition from a Single Image](https://mlsanity.com/paper/1911.09930/) (2019)
- [IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis](https://mlsanity.com/paper/2210.00647/) (2022)
- [Self-Supervised Intrinsic Image Decomposition](https://mlsanity.com/paper/1711.03678/) (2017)
- [Learning Non-Lambertian Object Intrinsics across ShapeNet Categories](https://mlsanity.com/paper/1612.08510/) (2016)
- [A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond](https://mlsanity.com/paper/2112.03842/) (2021)
- [Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression](https://mlsanity.com/paper/1512.02311/) (2015)
- [Joint Learning of Intrinsic Images and Semantic Segmentation](https://mlsanity.com/paper/1807.11857/) (2018)
- [PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition](https://mlsanity.com/paper/2203.16670/) (2022)
- [Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition](https://mlsanity.com/paper/1510.02413/) (2015)
- [CNN based Learning using Reflection and Retinex Models for Intrinsic Image Decomposition](https://mlsanity.com/paper/1712.01056/) (2017)
- [Reflectance Adaptive Filtering Improves Intrinsic Image Estimation](https://mlsanity.com/paper/1612.05062/) (2016)
- [Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields](https://mlsanity.com/paper/1603.06359/) (2016)
- [Intrinsic Image Decomposition using Paradigms](https://mlsanity.com/paper/2011.10512/) (2020)
- [ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition](https://mlsanity.com/paper/1912.04023/) (2019)
- [Intrinsic Image Transformation via Scale Space Decomposition](https://mlsanity.com/paper/1805.10253/) (2018)
- [Unsupervised Intrinsic Image Decomposition with LiDAR Intensity](https://mlsanity.com/paper/2303.10820/) (2023)
- [Point Cloud Color Constancy](https://mlsanity.com/paper/2111.11280/) (2021)

## Abstract (arXiv)

The purpose of intrinsic decomposition is to separate an image into its albedo (reflective properties) and shading components (illumination properties). This is challenging because it's an ill-posed problem. Conventional approaches primarily concentrate on 2D imagery and fail to fully exploit the capabilities of 3D data representation. 3D point clouds offer a more comprehensive format for representing scenes, as they combine geometric and color information effectively. To this end, in this paper, we introduce Point Intrinsic Net (PoInt-Net), which leverages 3D point cloud data to concurrently estimate albedo and shading maps. The merits of PoInt-Net include the following aspects. First, the model is efficient, achieving consistent performance across point clouds of any size with training only required on small-scale point clouds. Second, it exhibits remarkable robustness; even when trained exclusively on datasets comprising individual objects, PoInt-Net demonstrates strong generalization to unseen objects and scenes. Third, it delivers superior accuracy over conventional 2D approaches, demonstrating enhanced performance across various metrics on different datasets. (Code Released)