# End-to-end differentiable design of geometric waveguide displays

arXiv: https://arxiv.org/abs/2601.04370 · HTML: https://mlsanity.com/paper/2601.04370/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 30 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Builds on (most cited first, 3 of 3)

- [Curriculum Learning for ab initio Deep Learned Refractive Optics](https://mlsanity.com/paper/2302.01089/) (2023)
- [Neural 'Etendue Expander for Ultra-Wide-Angle High-Fidelity Holographic Display](https://mlsanity.com/paper/2109.08123/) (2021)
- [End-to-End Hybrid Refractive-Diffractive Lens Design with Differentiable Ray-Wave Model](https://mlsanity.com/paper/2406.00834/) (2024)

## Built on by (most cited first, 1 of 1)

- [Differentiable Polarized Path Tracing](https://mlsanity.com/paper/2607.13265/) (2026)

## Abstract (arXiv)

Geometric waveguides are a promising architecture for optical see-through augmented reality displays, but their performance is severely bottlenecked by the difficulty of jointly optimizing non-sequential light transport and polarization-dependent multilayer thin-film coatings. Here we present the first end-to-end differentiable optimization framework for geometric waveguide that couples non-sequential Monte Carlo polarization ray tracing with a differentiable transfer-matrix thin-film solver. A differentiable Monte Carlo ray tracer avoids the exponential growth of deterministic ray splitting while enabling gradients backpropagation from eyebox metrics to design parameters. With memory-saving strategies, we optimize more than one thousand layer-thickness parameters and billions of non-sequential ray-surface intersections on a single multi-GPU workstation. Automated layer pruning is achieved by starting from over-parameterized stacks and driving redundant layers to zero thickness under discrete manufacturability constraints, effectively performing topology optimization to discover optimal coating structures. On a representative design, starting from random initialization within thickness bounds, our method increases light efficiency from 4.1% to 33.5% and improves eyebox and FoV uniformity by ~17× and ~11×, respectively. Furthermore, we jointly optimize the waveguide and an image preprocessing network to improve perceived image quality. Our framework not only enables system-level, high-dimensional coating optimization inside the waveguide, but also expands the scope of differentiable optics for next-generation optical design.