# Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information

arXiv: https://arxiv.org/abs/2412.14603 · HTML: https://mlsanity.com/paper/2412.14603/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 33 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [Deep Optics for Monocular Depth Estimation and 3D Object Detection](https://mlsanity.com/paper/1904.08601/) (2019)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Curriculum Learning for ab initio Deep Learned Refractive Optics](https://mlsanity.com/paper/2302.01089/) (2023)
- [Fast Two-step Blind Optical Aberration Correction](https://mlsanity.com/paper/2208.00950/) (2022)
- [Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution](https://mlsanity.com/paper/2104.03078/) (2021)
- [Optical Aberration Correction in Postprocessing using Imaging Simulation](https://mlsanity.com/paper/2305.05867/) (2023)
- [The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection](https://mlsanity.com/paper/2212.04441/) (2022)
- [Computational Optics for Mobile Terminals in Mass Production](https://mlsanity.com/paper/2305.05886/) (2023)
- [Revealing the preference for correcting separated aberrations in joint optic-image design](https://mlsanity.com/paper/2309.04342/) (2023)

## Built on by (most cited first, 3 of 3)

- [Direct Kernel Optimization: Efficient Design for Opto-Electronic Convolutional Neural Networks](https://mlsanity.com/paper/2511.02065/) (2025)
- [CircleFlow: Flow-Guided Camera Blur Estimation using a Circle Grid Target](https://mlsanity.com/paper/2512.00796/) (2025)
- [Beyond Bayer: Task-Optimal Sensor Co-Design for Robust Autonomous-Driving Segmentation](https://mlsanity.com/paper/2606.24096/) (2026)

## Abstract (arXiv)

Recently, the joint design of optical systems and downstream algorithms is showing significant potential. However, existing rays-described methods are limited to optimizing geometric degradation, making it difficult to fully represent the optical characteristics of complex, miniaturized lenses constrained by wavefront aberration or diffraction effects. In this work, we introduce a precise optical simulation model, and every operation in pipeline is differentiable. This model employs a novel initial value strategy to enhance the reliability of intersection calculation on high aspherics. Moreover, it utilizes a differential operator to reduce memory consumption during coherent point spread function calculations. To efficiently address various degradation, we design a joint optimization procedure that leverages field information. Guided by a general restoration network, the proposed method not only enhances the image quality, but also successively improves the optical performance across multiple lenses that are already in professional level. This joint optimization pipeline offers innovative insights into the practical design of sophisticated optical systems and post-processing algorithms. The source code will be made publicly available at https://github.com/Zrr-ZJU/Successive-optimization