# Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation

arXiv: https://arxiv.org/abs/2405.03662 · HTML: https://mlsanity.com/paper/2405.03662/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 20 of 20)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping](https://mlsanity.com/paper/2105.07014/) (2021)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution](https://mlsanity.com/paper/2011.04566/) (2020)
- [Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model](https://mlsanity.com/paper/2207.10040/) (2022)
- [Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulence](https://mlsanity.com/paper/2009.00071/) (2020)
- [Simulating Anisoplanatic Turbulence by Sampling Inter-modal and Spatially Correlated Zernike Coefficients](https://mlsanity.com/paper/2004.11210/) (2020)
- [Physics-Driven Turbulence Image Restoration with Stochastic Refinement](https://mlsanity.com/paper/2307.10603/) (2023)
- [Learning to restore images degraded by atmospheric turbulence using uncertainty](https://mlsanity.com/paper/2207.03447/) (2022)
- [Subsampled Turbulence Removal Network](https://mlsanity.com/paper/1807.04418/) (2018)
- [Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Network](https://mlsanity.com/paper/2204.06989/) (2022)
- [Restoration of Atmospheric Turbulence-distorted Images via RPCA and Quasiconformal Maps](https://mlsanity.com/paper/1704.03140/) (2017)
- [Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model](https://mlsanity.com/paper/2207.06377/) (2022)
- [Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence](https://mlsanity.com/paper/2210.06713/) (2022)
- [Flow-Guided Video Inpainting with Scene Templates](https://mlsanity.com/paper/2108.12845/) (2021)
- [TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulence](https://mlsanity.com/paper/2203.06764/) (2022)

## Built on by (most cited first, 6 of 6)

- [Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation](https://mlsanity.com/paper/2504.02697/) (2025)
- [Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens](https://mlsanity.com/paper/2508.04928/) (2025)
- [ETA: Energy-based Test-time Adaptation for Depth Completion](https://mlsanity.com/paper/2508.05989/) (2025)
- [Triangular Consistency as a Universal Constraint for Learning Optical Flow](https://mlsanity.com/paper/2606.19938/) (2026)
- [VisTa3D: A Dataset and Benchmark for Thin Object Reconstruction from Vision, Tactile, and 3D Point Clouds](https://mlsanity.com/paper/2608.20740/) (2026)
- [Circular Quasiconformal Deturbulence: Geometry-Based Restoration from Multiple Turbulent Frames](https://mlsanity.com/paper/2504.13432/) (2025)

## Abstract (arXiv)

We describe a method for recovering the irradiance underlying a collection of images corrupted by atmospheric turbulence. Since supervised data is often technically impossible to obtain, assumptions and biases have to be imposed to solve this inverse problem, and we choose to model them explicitly. Rather than initializing a latent irradiance ("template") by heuristics to estimate deformation, we select one of the images as a reference, and model the deformation in this image by the aggregation of the optical flow from it to other images, exploiting a prior imposed by Central Limit Theorem. Then with a novel flow inversion module, the model registers each image TO the template but WITHOUT the template, avoiding artifacts related to poor template initialization. To illustrate the robustness of the method, we simply (i) select the first frame as the reference and (ii) use the simplest optical flow to estimate the warpings, yet the improvement in registration is decisive in the final reconstruction, as we achieve state-of-the-art performance despite its simplicity. The method establishes a strong baseline that can be further improved by integrating it seamlessly into more sophisticated pipelines, or with domain-specific methods if so desired.