# Curriculum Learning for ab initio Deep Learned Refractive Optics

arXiv: https://arxiv.org/abs/2302.01089 · HTML: https://mlsanity.com/paper/2302.01089/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 12 of 12)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Deep Optics for Monocular Depth Estimation and 3D Object Detection](https://mlsanity.com/paper/1904.08601/) (2019)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics](https://mlsanity.com/paper/2009.00463/) (2020)
- [Light Stage Super-Resolution: Continuous High-Frequency Relighting](https://mlsanity.com/paper/2010.08888/) (2020)
- [The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection](https://mlsanity.com/paper/2212.04441/) (2022)
- [Mask-ToF: Learning Microlens Masks for Flying Pixel Correction in Time-of-Flight Imaging](https://mlsanity.com/paper/2103.16693/) (2021)
- [Hybrid Diffractive Optics Design via Hardware-in-the-Loop Methodology for Achromatic Extended-Depth-of-Field Imaging](https://mlsanity.com/paper/2203.16407/) (2022)
- [Learning Wavefront Coding for Extended Depth of Field Imaging](https://mlsanity.com/paper/1912.13423/) (2019)

## Built on by (most cited first, 30 of 32)

- [Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers](https://mlsanity.com/paper/2306.12992/) (2023)
- [Aberration-Aware Depth-from-Focus](https://mlsanity.com/paper/2303.04654/) (2023)
- [End-to-End Hybrid Refractive-Diffractive Lens Design with Differentiable Ray-Wave Model](https://mlsanity.com/paper/2406.00834/) (2024)
- [Task-Driven Lens Design](https://mlsanity.com/paper/2305.17185/) (2023)
- [Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems](https://mlsanity.com/paper/2404.19201/) (2024)
- [Revealing the preference for correcting separated aberrations in joint optic-image design](https://mlsanity.com/paper/2309.04342/) (2023)
- [OmniLens: Towards Universal Lens Aberration Correction via LensLib-to-Specific Domain Adaptation](https://mlsanity.com/paper/2409.05809/) (2024)
- [Efficient Depth- and Spatially-Varying Image Simulation for Defocus Deblur](https://mlsanity.com/paper/2507.00372/) (2025)
- [Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges](https://mlsanity.com/paper/2402.12041/) (2024)
- [Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information](https://mlsanity.com/paper/2412.14603/) (2024)
- [OPTIAGENT: A Physics-Driven Agentic Framework for Automated Optical Design](https://mlsanity.com/paper/2602.23761/) (2026)
- [TaCOS: Task-Specific Camera Optimization with Simulation](https://mlsanity.com/paper/2404.11031/) (2024)
- [EventTracer: Fast Path Tracing-based Event Stream Rendering](https://mlsanity.com/paper/2508.18071/) (2025)
- [Neuro-inspired automated lens design](https://mlsanity.com/paper/2510.09979/) (2025)
- [Socratic agents for autonomous scientific discovery in high-dimensional physical systems](https://mlsanity.com/paper/2606.26722/) (2026)
- [Simulating Dual-Pixel Images From Ray Tracing For Depth Estimation](https://mlsanity.com/paper/2503.11213/) (2025)
- [Fovea Stacking: Imaging with Dynamic Localized Aberration Correction](https://mlsanity.com/paper/2506.00716/) (2025)
- [Collaborative On-Sensor Array Cameras](https://mlsanity.com/paper/2506.04061/) (2025)
- [Learned Off-aperture Encoding for Wide Field-of-view RGBD Imaging](https://mlsanity.com/paper/2507.22523/) (2025)
- [End-to-end differentiable design of geometric waveguide displays](https://mlsanity.com/paper/2601.04370/) (2026)
- [Neural Dynamic Data Valuation via Stochastic State-Adjoint Trajectories](https://mlsanity.com/paper/2404.19557/) (2024)
- [Large-Area Fabrication-Aware Computational Diffractive Optics](https://mlsanity.com/paper/2505.22313/) (2025)
- [Automated design of compound lenses with discrete-continuous optimization](https://mlsanity.com/paper/2509.23572/) (2025)
- [JOCA: Task-Driven Joint Optimisation of Camera Hardware and Adaptive Camera Control Algorithms](https://mlsanity.com/paper/2512.06763/) (2025)
- [Graph Neural Networks for Interferometer Simulations](https://mlsanity.com/paper/2512.16051/) (2025)
- [Learning to Sense for Driving: Joint Optics-Sensor-Model Co-Design for Semantic Segmentation](https://mlsanity.com/paper/2512.20815/) (2025)
- [Towards Universal Computational Aberration Correction in Photographic Cameras: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/2603.12083/) (2026)
- [A Differentiable Ray-Wave Framework for Hybrid Refractive-Diffractive System Modeling and Optimization](https://mlsanity.com/paper/2605.15418/) (2026)
- [Beyond Bayer: Task-Optimal Sensor Co-Design for Robust Autonomous-Driving Segmentation](https://mlsanity.com/paper/2606.24096/) (2026)
- [Realistic Compound-Lens Defocus Blur Synthesis](https://mlsanity.com/paper/2607.05837/) (2026)

## Abstract (arXiv)

Deep optical optimization has recently emerged as a new paradigm for designing computational imaging systems using only the output image as the objective. However, it has been limited to either simple optical systems consisting of a single element such as a diffractive optical element (DOE) or metalens, or the fine-tuning of compound lenses from good initial designs. Here we present a DeepLens design method based on curriculum learning, which is able to learn optical designs of compound lenses ab initio from randomly initialized surfaces without human intervention, therefore overcoming the need for a good initial design. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens in a cellphone-style form factor, with highly aspheric surfaces and a short back focal length.