# PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction

arXiv: https://arxiv.org/abs/2310.07504 · HTML: https://mlsanity.com/paper/2310.07504/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1704.02422/) (2017)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Phase Retrieval via Wirtinger Flow: Theory and Algorithms](https://mlsanity.com/paper/1407.1065/) (2014)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Online Deep Equilibrium Learning for Regularization by Denoising](https://mlsanity.com/paper/2205.13051/) (2022)
- [SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees](https://mlsanity.com/paper/2101.09379/) (2021)
- [SHARP: a distributed, GPU-based ptychographic solver](https://mlsanity.com/paper/1602.01448/) (2016)
- [Accelerated Wirtinger Flow: A fast algorithm for ptychography](https://mlsanity.com/paper/1806.05546/) (2018)
- [Compressive Ptychography using Deep Image and Generative Priors](https://mlsanity.com/paper/2205.02397/) (2022)
- [A parameter refinement method for Ptychography based on Deep Learning concepts](https://mlsanity.com/paper/2105.08058/) (2021)
- [Projected Multi-Agent Consensus Equilibrium for Ptychographic Image Reconstruction](https://mlsanity.com/paper/2111.14240/) (2021)
- [Deep Iterative Phase Retrieval for Ptychography](https://mlsanity.com/paper/2202.10573/) (2022)
- [Projected Multi-Agent Consensus Equilibrium (PMACE) with Application to Ptychography](https://mlsanity.com/paper/2303.15679/) (2023)
- [LWGNet: Learned Wirtinger Gradients for Fourier Ptychographic Phase Retrieval](https://mlsanity.com/paper/2208.04283/) (2022)
- [Unrolled Wirtinger Flow with Deep Decoding Priors for Phaseless Imaging](https://mlsanity.com/paper/2108.01735/) (2021)

## Built on by (most cited first, 9 of 9)

- [PtychoFormer: A Transformer-based Model for Ptychographic Phase Retrieval](https://mlsanity.com/paper/2410.17377/) (2024)
- [A Physics-Inspired Deep Learning Framework with Polar Coordinate Attention for Ptychographic Imaging](https://mlsanity.com/paper/2412.06806/) (2024)
- [Ptychographic Image Reconstruction from Limited Data via Score-Based Diffusion Models with Physics-Guidance](https://mlsanity.com/paper/2502.18767/) (2025)
- [Improving Multislice Electron Ptychography with a Generative Prior](https://mlsanity.com/paper/2507.17800/) (2025)
- [Comprehensive Examination of Unrolled Networks for Solving Linear Inverse Problems](https://mlsanity.com/paper/2501.04608/) (2025)
- [Learning neural representations for X-ray ptychography reconstruction with unknown probes](https://mlsanity.com/paper/2509.04402/) (2025)
- [Fidelity-preserving enhancement of ptychography with foundational text-to-image models](https://mlsanity.com/paper/2509.04513/) (2025)
- [Towards generalizable deep ptychography neural networks](https://mlsanity.com/paper/2509.25104/) (2025)
- [Contrast-invariant deep ptychography neural networks](https://mlsanity.com/paper/2608.02869/) (2026)

## Abstract (arXiv)

Ptychography is an imaging technique that captures multiple overlapping snapshots of a sample, illuminated coherently by a moving localized probe. The image recovery from ptychographic data is generally achieved via an iterative algorithm that solves a nonlinear phase retrieval problem derived from measured diffraction patterns. However, these iterative approaches have high computational cost. In this paper, we introduce PtychoDV, a novel deep model-based network designed for efficient, high-quality ptychographic image reconstruction. PtychoDV comprises a vision transformer that generates an initial image from the set of raw measurements, taking into consideration their mutual correlations. This is followed by a deep unrolling network that refines the initial image using learnable convolutional priors and the ptychography measurement model. Experimental results on simulated data demonstrate that PtychoDV is capable of outperforming existing deep learning methods for this problem, and significantly reduces computational cost compared to iterative methodologies, while maintaining competitive performance.