# PtychoFormer: A Transformer-based Model for Ptychographic Phase Retrieval

arXiv: https://arxiv.org/abs/2410.17377 · HTML: https://mlsanity.com/paper/2410.17377/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [How Much Position Information Do Convolutional Neural Networks Encode?](https://mlsanity.com/paper/2001.08248/) (2020)
- [Deep learning at the edge enables real-time streaming ptychographic imaging](https://mlsanity.com/paper/2209.09408/) (2022)
- [PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction](https://mlsanity.com/paper/2310.07504/) (2023)
- [Real-time sparse-sampled Ptychographic imaging through deep neural networks](https://mlsanity.com/paper/2004.08247/) (2020)
- [Ptychographic reconstruction with wavefront initialization](https://mlsanity.com/paper/2205.11996/) (2022)

## Built on by (most cited first, 5 of 5)

- [A Physics-Inspired Deep Learning Framework with Polar Coordinate Attention for Ptychographic Imaging](https://mlsanity.com/paper/2412.06806/) (2024)
- [Towards generalizable deep ptychography neural networks](https://mlsanity.com/paper/2509.25104/) (2025)
- [Circular Phase Representation and Geometry-Aware Optimization for Ptychographic Image Reconstruction](https://mlsanity.com/paper/2604.26664/) (2026)
- [Contrast-invariant deep ptychography neural networks](https://mlsanity.com/paper/2608.02869/) (2026)
- [Zero-Shot Cross-Material Ptychographic Phase Reconstruction Using Deep Learning](https://mlsanity.com/paper/2609.13969/) (2026)

## Abstract (arXiv)

Ptychography is a computational method of microscopy that recovers high-resolution transmission images of samples from a series of diffraction patterns. While conventional phase retrieval algorithms can iteratively recover the images, they require oversampled diffraction patterns, incur significant computational costs, and struggle to recover the absolute phase of the sample's transmission function. Deep learning algorithms for ptychography are a promising approach to resolving the limitations of iterative algorithms. We present PtychoFormer, a hierarchical transformer-based model for data-driven single-shot ptychographic phase retrieval. PtychoFormer processes subsets of diffraction patterns, generating local inferences that are seamlessly stitched together to produce a high-quality reconstruction. Our model exhibits tolerance to sparsely scanned diffraction patterns and achieves up to 3600 times faster imaging speed than the extended ptychographic iterative engine (ePIE). We also propose the extended-PtychoFormer (ePF), a hybrid approach that combines the benefits of PtychoFormer with the ePIE. ePF minimizes global phase shifts and significantly enhances reconstruction quality, achieving state-of-the-art phase retrieval in ptychography.