# Differentiable Adaptive 4D Structured Illumination for Joint Capture of Shape and Reflectance

arXiv: https://arxiv.org/abs/2605.06214 · HTML: https://mlsanity.com/paper/2605.06214/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 47 references that are in the map.
- First posted on arXiv 7 May 2026.

## Builds on (most cited first, 8 of 8)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [GS^3: Efficient Relighting with Triple Gaussian Splatting](https://mlsanity.com/paper/2410.11419/) (2024)
- [Relighting Neural Radiance Fields with Shadow and Highlight Hints](https://mlsanity.com/paper/2308.13404/) (2023)
- [Learning to Learn and Sample BRDFs](https://mlsanity.com/paper/2210.03510/) (2022)
- [Single-Photon Structured Light](https://mlsanity.com/paper/2204.05300/) (2022)
- [Differentiable Display Photometric Stereo](https://mlsanity.com/paper/2306.13325/) (2023)
- [Depth Reconstruction with Neural Signed Distance Fields in Structured Light Systems](https://mlsanity.com/paper/2405.12006/) (2024)
- [Efficient Reflectance Capture with a Deep Gated Mixture-of-Experts](https://mlsanity.com/paper/2203.15258/) (2022)

## Abstract (arXiv)

We present a differentiable framework to adaptively compute 4D illumination conditions with respect to an object, for efficient, high-quality simultaneous acquisition of its shape and reflectance, with a unified spatial-angular structured light and a single camera. Using a simple histogram-based pixel-level probability model for depth and reflectance, we differentiably link the next illumination condition(s) with a loss that encourages the reduction in depth uncertainty. As new structured illumination is cast, corresponding image measurements are used to update the uncertainty at each pixel. Finally, a fine-tuning-based approach reconstructs the depth map and reflectance parameter maps, by minimizing the differences between all physical measurements and their simulated counterparts. The effectiveness of our framework is demonstrated on physical objects with wide variations in shape and appearance. Our depth results compare favorably with state-of-the-art techniques, while our reflectance results are comparable when validated against photographs.