# Physics-Driven Turbulence Image Restoration with Stochastic Refinement

arXiv: https://arxiv.org/abs/2307.10603 · HTML: https://mlsanity.com/paper/2307.10603/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 73 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 30 of 43)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [Parallel Diffusion Models of Operator and Image for Blind Inverse Problems](https://mlsanity.com/paper/2211.10656/) (2022)
- [Detecting Text in Natural Image with Connectionist Text Proposal Network](https://mlsanity.com/paper/1609.03605/) (2016)
- [Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Dataset](https://mlsanity.com/paper/2211.01917/) (2022)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution](https://mlsanity.com/paper/2011.04566/) (2020)
- [Evaluating Robustness of Deep Image Super-Resolution against Adversarial Attacks](https://mlsanity.com/paper/1904.06097/) (2019)
- [Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model](https://mlsanity.com/paper/2207.10040/) (2022)
- [AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2208.11284/) (2022)

## Built on by (most cited first, 24 of 24)

- [Spatio-Temporal Turbulence Mitigation: A Translational Perspective](https://mlsanity.com/paper/2401.04244/) (2024)
- [Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait](https://mlsanity.com/paper/2505.04616/) (2025)
- [Deep Learning Techniques for Atmospheric Turbulence Removal: A Review](https://mlsanity.com/paper/2409.14587/) (2024)
- [Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation](https://mlsanity.com/paper/2504.02697/) (2025)
- [Deformation-Invariant Neural Network and Its Applications in Distorted Image Restoration and Analysis](https://mlsanity.com/paper/2310.02641/) (2023)
- [Long-range Turbulence Mitigation: A Large-scale Dataset and A Coarse-to-fine Framework](https://mlsanity.com/paper/2407.08377/) (2024)
- [Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation](https://mlsanity.com/paper/2405.03662/) (2024)
- [DeTurb: Atmospheric Turbulence Mitigation with Deformable 3D Convolutions and 3D Swin Transformers](https://mlsanity.com/paper/2407.20855/) (2024)
- [Real-World Atmospheric Turbulence Correction via Domain Adaptation](https://mlsanity.com/paper/2402.07371/) (2024)
- [EGTM: Event-guided Efficient Turbulence Mitigation](https://mlsanity.com/paper/2509.03808/) (2025)
- [DMAT: An End-to-End Framework for Joint Atmospheric Turbulence Mitigation and Object Detection](https://mlsanity.com/paper/2507.04323/) (2025)
- [Astrophotography turbulence mitigation via generative models](https://mlsanity.com/paper/2506.02981/) (2025)
- [JDATT: A Joint Distillation Framework for Atmospheric Turbulence Mitigation and Target Detection](https://mlsanity.com/paper/2507.19780/) (2025)
- [RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator](https://mlsanity.com/paper/2508.11409/) (2025)
- [Continuous Exposure-Time Modeling for Realistic Atmospheric Turbulence Synthesis](https://mlsanity.com/paper/2603.01398/) (2026)
- [Probabilistic Prior Driven Attention Mechanism Based on Diffusion Model for Imaging Through Atmospheric Turbulence](https://mlsanity.com/paper/2411.10321/) (2024)
- [Optical aberrations in autonomous driving: Physics-informed parameterized temperature scaling for neural network uncertainty calibration](https://mlsanity.com/paper/2412.13695/) (2024)
- [prNet: Data-Driven Phase Retrieval via Stochastic Refinement](https://mlsanity.com/paper/2507.09608/) (2025)
- [Active Convolved Illumination with Deep Transfer Learning for Complex Beam Transmission through Atmospheric Turbulence](https://mlsanity.com/paper/2512.19540/) (2025)
- [High-Quality and Efficient Turbulence Mitigation with Events](https://mlsanity.com/paper/2603.20708/) (2026)
- [A unified Benchmark for Multi-Frame Image Restoration under Severe Refractive Warping](https://mlsanity.com/paper/2605.05079/) (2026)
- [ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes](https://mlsanity.com/paper/2605.21440/) (2026)
- [D2Turb: Depth-Aware Simulation and Decoupled Learning for Single-Frame Atmospheric Turbulence Mitigation](https://mlsanity.com/paper/2605.27460/) (2026)
- [Restore What Matters: Lessons from Joint Restoration and Recognition](https://mlsanity.com/paper/2609.13791/) (2026)

## Abstract (arXiv)

Image distortion by atmospheric turbulence is a stochastic degradation, which is a critical problem in long-range optical imaging systems. A number of research has been conducted during the past decades, including model-based and emerging deep-learning solutions with the help of synthetic data. Although fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions recently, the training of such models only relies on the synthetic data and ground truth pairs. This paper proposes the Physics-integrated Restoration Network (PiRN) to bring the physics-based simulator directly into the training process to help the network to disentangle the stochasticity from the degradation and the underlying image. Furthermore, to overcome the ``average effect" introduced by deterministic models and the domain gap between the synthetic and real-world degradation, we further introduce PiRN with Stochastic Refinement (PiRN-SR) to boost its perceptual quality. Overall, our PiRN and PiRN-SR improve the generalization to real-world unknown turbulence conditions and provide a state-of-the-art restoration in both pixel-wise accuracy and perceptual quality. Our codes are available at urlhttps://github.com/VITA-Group/PiRN.