# Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data

arXiv: https://arxiv.org/abs/2309.06302 · HTML: https://mlsanity.com/paper/2309.06302/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 8 of 8)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [A Dataset of Multi-Illumination Images in the Wild](https://mlsanity.com/paper/1910.08131/) (2019)
- [Learning Non-Lambertian Object Intrinsics across ShapeNet Categories](https://mlsanity.com/paper/1612.08510/) (2016)
- [Reduce Information Loss in Transformers for Pluralistic Image Inpainting](https://mlsanity.com/paper/2205.05076/) (2022)
- [Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning](https://mlsanity.com/paper/2211.14751/) (2022)
- [Leveraging Multi-view Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation](https://mlsanity.com/paper/1911.07262/) (2019)

## Built on by (most cited first, 5 of 5)

- [Diversify, Don't Fine-Tune: Scaling Up Visual Recognition Training with Synthetic Images](https://mlsanity.com/paper/2312.02253/) (2023)
- [Single Document Image Highlight Removal via A Large-Scale Real-World Dataset and A Location-Aware Network](https://mlsanity.com/paper/2504.14238/) (2025)
- [Exploring Spectral Characteristics for Single Image Reflection Removal](https://mlsanity.com/paper/2509.12627/) (2025)
- [Shift-Window Meets Dual Attention: A Multi-Model Architecture for Specular Highlight Removal](https://mlsanity.com/paper/2512.04496/) (2025)
- [3DReflecNet: A Large-Scale Dataset for 3D Reconstruction of Reflective, Transparent, and Low-Texture Objects](https://mlsanity.com/paper/2605.10204/) (2026)

## Abstract (arXiv)

This paper aims to remove specular highlights from a single object-level image. Although previous methods have made some progresses, their performance remains somewhat limited, particularly for real images with complex specular highlights. To this end, we propose a three-stage network to address them. Specifically, given an input image, we first decompose it into the albedo, shading, and specular residue components to estimate a coarse specular-free image. Then, we further refine the coarse result to alleviate its visual artifacts such as color distortion. Finally, we adjust the tone of the refined result to match that of the input as closely as possible. In addition, to facilitate network training and quantitative evaluation, we present a large-scale synthetic dataset of object-level images, covering diverse objects and illumination conditions. Extensive experiments illustrate that our network is able to generalize well to unseen real object-level images, and even produce good results for scene-level images with multiple background objects and complex lighting.