# Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo

arXiv: https://arxiv.org/abs/2011.13117 · HTML: https://mlsanity.com/paper/2011.13117/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [A Survey on Deep Stereo Matching in the Twenties](https://mlsanity.com/paper/2407.07816/) (2024)
- [Thin On-Sensor Nanophotonic Array Cameras](https://mlsanity.com/paper/2308.02797/) (2023)
- [CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo](https://mlsanity.com/paper/2104.04641/) (2021)
- [DISeR: Designing Imaging Systems with Reinforcement Learning](https://mlsanity.com/paper/2309.13851/) (2023)
- [Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection](https://mlsanity.com/paper/2204.03929/) (2022)
- [Differentiable Display Photometric Stereo](https://mlsanity.com/paper/2306.13325/) (2023)
- [Dispersed Structured Light for Hyperspectral 3D Imaging](https://mlsanity.com/paper/2311.18287/) (2023)
- [Optimizing Camera Configurations for Multi-View Pedestrian Detection](https://mlsanity.com/paper/2312.02144/) (2023)
- [Dense Dispersed Structured Light for Hyperspectral 3D Imaging of Dynamic Scenes](https://mlsanity.com/paper/2412.01140/) (2024)
- [TaCOS: Task-Specific Camera Optimization with Simulation](https://mlsanity.com/paper/2404.11031/) (2024)
- [SpectralZoom: Efficient Segmentation with an Adaptive Hyperspectral Camera](https://mlsanity.com/paper/2406.04287/) (2024)
- [LED: Light Enhanced Depth Estimation at Night](https://mlsanity.com/paper/2409.08031/) (2024)
- [Robust Single-shot Structured Light 3D Imaging via Neural Feature Decoding](https://mlsanity.com/paper/2512.14028/) (2025)
- [SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing](https://mlsanity.com/paper/2103.12981/) (2021)
- [Seeing Far in the Dark with Patterned Flash](https://mlsanity.com/paper/2207.12570/) (2022)
- [ActiveNeuS: Neural Signed Distance Fields for Active Stereo](https://mlsanity.com/paper/2410.15376/) (2024)
- [Collaborative On-Sensor Array Cameras](https://mlsanity.com/paper/2506.04061/) (2025)
- [Neural 360^circ Structured Light with Learned Metasurfaces](https://mlsanity.com/paper/2306.13361/) (2023)
- [JOCA: Task-Driven Joint Optimisation of Camera Hardware and Adaptive Camera Control Algorithms](https://mlsanity.com/paper/2512.06763/) (2025)
- [Broadband Hyperspectral 3D Imaging using Dispersed Structured Light](https://mlsanity.com/paper/2605.25757/) (2026)
- [NSL-SLAM: High-Fidelity Neural Structured-Light Depth for Practical SLAM and Reconstruction](https://mlsanity.com/paper/2607.24495/) (2026)

## Abstract (arXiv)

Active stereo cameras that recover depth from structured light captures have become a cornerstone sensor modality for 3D scene reconstruction and understanding tasks across application domains. Existing active stereo cameras project a pseudo-random dot pattern on object surfaces to extract disparity independently of object texture. Such hand-crafted patterns are designed in isolation from the scene statistics, ambient illumination conditions, and the reconstruction method. In this work, we propose the first method to jointly learn structured illumination and reconstruction, parameterized by a diffractive optical element and a neural network, in an end-to-end fashion. To this end, we introduce a novel differentiable image formation model for active stereo, relying on both wave and geometric optics, and a novel trinocular reconstruction network. The jointly optimized pattern, which we dub "Polka Lines," together with the reconstruction network, achieve state-of-the-art active-stereo depth estimates across imaging conditions. We validate the proposed method in simulation and on a hardware prototype, and show that our method outperforms existing active stereo systems.