# What is Wrong with End-to-End Learning for Phase Retrieval?

arXiv: https://arxiv.org/abs/2403.15448 · HTML: https://mlsanity.com/paper/2403.15448/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 21 of 21)

- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Learning two-phase microstructure evolution using neural operators and autoencoder architectures](https://mlsanity.com/paper/2204.07230/) (2022)
- [The gap between theory and practice in function approximation with deep neural networks](https://mlsanity.com/paper/2001.07523/) (2020)
- [prDeep: Robust Phase Retrieval with a Flexible Deep Network](https://mlsanity.com/paper/1803.00212/) (2018)
- [Lensless computational imaging through deep learning](https://mlsanity.com/paper/1702.08516/) (2017)
- [Deep Non-Rigid Structure from Motion](https://mlsanity.com/paper/1908.00052/) (2019)
- [KBNet: Kernel Basis Network for Image Restoration](https://mlsanity.com/paper/2303.02881/) (2023)
- [FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image](https://mlsanity.com/paper/1903.12305/) (2019)
- [Low Photon Count Phase Retrieval Using Deep Learning](https://mlsanity.com/paper/1806.10029/) (2018)
- [Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media](https://mlsanity.com/paper/1806.04139/) (2018)
- [Real-time coherent diffraction inversion using deep generative networks](https://mlsanity.com/paper/1806.03992/) (2018)
- [Deep Iterative Reconstruction for Phase Retrieval](https://mlsanity.com/paper/1904.11301/) (2019)
- [Phase Retrieval Using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1912.04981/) (2019)
- [Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild](https://mlsanity.com/paper/2001.10090/) (2020)
- [Inverse Problems, Deep Learning, and Symmetry Breaking](https://mlsanity.com/paper/2003.09077/) (2020)
- [SiSPRNet: End-to-End Learning for Single-Shot Phase Retrieval](https://mlsanity.com/paper/2205.11434/) (2022)
- [Egocentric Scene Understanding via Multimodal Spatial Rectifier](https://mlsanity.com/paper/2207.07077/) (2022)
- [ADMM based Fourier phase retrieval with untrained generative prior](https://mlsanity.com/paper/2210.12646/) (2022)

## Built on by (most cited first, 2 of 2)

- [FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2508.00721/) (2025)
- [Saving Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2511.16520/) (2025)

## Abstract (arXiv)

For nonlinear inverse problems that are prevalent in imaging science, symmetries in the forward model are common. When data-driven deep learning approaches are used to solve such problems, these intrinsic symmetries can cause substantial learning difficulties. In this paper, we explain how such difficulties arise and, more importantly, how to overcome them by preprocessing the training set before any learning, i.e., symmetry breaking. We take far-field phase retrieval (FFPR), which is central to many areas of scientific imaging, as an example and show that symmetric breaking can substantially improve data-driven learning. We also formulate the mathematical principle of symmetry breaking.