# Imaging through the Atmosphere using Turbulence Mitigation Transformer

arXiv: https://arxiv.org/abs/2207.06465 · HTML: https://mlsanity.com/paper/2207.06465/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 81 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Understanding The Robustness in Vision Transformers](https://mlsanity.com/paper/2204.12451/) (2022)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [Evaluating Robustness of Deep Image Super-Resolution against Adversarial Attacks](https://mlsanity.com/paper/1904.06097/) (2019)
- [Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulence](https://mlsanity.com/paper/2009.00071/) (2020)
- [Learning to restore images degraded by atmospheric turbulence using uncertainty](https://mlsanity.com/paper/2207.03447/) (2022)
- [Subsampled Turbulence Removal Network](https://mlsanity.com/paper/1807.04418/) (2018)
- [Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Network](https://mlsanity.com/paper/2204.06989/) (2022)
- [On the Generalization of BasicVSR++ to Video Deblurring and Denoising](https://mlsanity.com/paper/2204.05308/) (2022)
- [Restoration of Atmospheric Turbulence-distorted Images via RPCA and Quasiconformal Maps](https://mlsanity.com/paper/1704.03140/) (2017)
- [Multi-Scale Memory-Based Video Deblurring](https://mlsanity.com/paper/2204.02977/) (2022)

## Built on by (most cited first, 30 of 31)

- [Physics-Driven Turbulence Image Restoration with Stochastic Refinement](https://mlsanity.com/paper/2307.10603/) (2023)
- [Spatio-Temporal Turbulence Mitigation: A Translational Perspective](https://mlsanity.com/paper/2401.04244/) (2024)
- [Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait](https://mlsanity.com/paper/2505.04616/) (2025)
- [Deep Learning Techniques for Atmospheric Turbulence Removal: A Review](https://mlsanity.com/paper/2409.14587/) (2024)
- [Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation](https://mlsanity.com/paper/2504.02697/) (2025)
- [Long-range Turbulence Mitigation: A Large-scale Dataset and A Coarse-to-fine Framework](https://mlsanity.com/paper/2407.08377/) (2024)
- [Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence](https://mlsanity.com/paper/2404.13605/) (2024)
- [DeTurb: Atmospheric Turbulence Mitigation with Deformable 3D Convolutions and 3D Swin Transformers](https://mlsanity.com/paper/2407.20855/) (2024)
- [Whitened Score Diffusion: A Structured Prior for Imaging Inverse Problems](https://mlsanity.com/paper/2505.10311/) (2025)
- [MAMAT: 3D Mamba-Based Atmospheric Turbulence Removal and its Object Detection Capability](https://mlsanity.com/paper/2503.17700/) (2025)
- [Weakly Supervised Face and Whole Body Recognition in Turbulent Environments](https://mlsanity.com/paper/2308.11757/) (2023)
- [Real-World Atmospheric Turbulence Correction via Domain Adaptation](https://mlsanity.com/paper/2402.07371/) (2024)
- [EvTurb: Event Camera Guided Turbulence Removal](https://mlsanity.com/paper/2508.10582/) (2025)
- [EGTM: Event-guided Efficient Turbulence Mitigation](https://mlsanity.com/paper/2509.03808/) (2025)
- [NeRT: Implicit Neural Representations for General Unsupervised Turbulence Mitigation](https://mlsanity.com/paper/2308.00622/) (2023)
- [Source-free Domain Adaptation for Video Object Detection Under Adverse Image Conditions](https://mlsanity.com/paper/2404.15252/) (2024)
- [DMAT: An End-to-End Framework for Joint Atmospheric Turbulence Mitigation and Object Detection](https://mlsanity.com/paper/2507.04323/) (2025)
- [Astrophotography turbulence mitigation via generative models](https://mlsanity.com/paper/2506.02981/) (2025)
- [JDATT: A Joint Distillation Framework for Atmospheric Turbulence Mitigation and Target Detection](https://mlsanity.com/paper/2507.19780/) (2025)
- [RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator](https://mlsanity.com/paper/2508.11409/) (2025)
- [Continuous Exposure-Time Modeling for Realistic Atmospheric Turbulence Synthesis](https://mlsanity.com/paper/2603.01398/) (2026)
- [1st Solution Places for CVPR 2023 UG2+ Challenge Track 2.2-Coded Target Restoration through Atmospheric Turbulence](https://mlsanity.com/paper/2306.09379/) (2023)
- [Unsupervised Region-Growing Network for Object Segmentation in Atmospheric Turbulence](https://mlsanity.com/paper/2311.03572/) (2023)
- [ConVRT: Consistent Video Restoration Through Turbulence with Test-time Optimization of Neural Video Representations](https://mlsanity.com/paper/2312.04679/) (2023)
- [Active Convolved Illumination with Deep Transfer Learning for Complex Beam Transmission through Atmospheric Turbulence](https://mlsanity.com/paper/2512.19540/) (2025)
- [GSTurb: Gaussian Splatting for Atmospheric Turbulence Mitigation](https://mlsanity.com/paper/2602.22800/) (2026)
- [High-speed Imaging through Turbulence with Event-based Light Fields](https://mlsanity.com/paper/2603.14023/) (2026)
- [High-Quality and Efficient Turbulence Mitigation with Events](https://mlsanity.com/paper/2603.20708/) (2026)
- [ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes](https://mlsanity.com/paper/2605.21440/) (2026)
- [D2Turb: Depth-Aware Simulation and Decoupled Learning for Single-Frame Atmospheric Turbulence Mitigation](https://mlsanity.com/paper/2605.27460/) (2026)

## Abstract (arXiv)

Restoring images distorted by atmospheric turbulence is a ubiquitous problem in long-range imaging applications. While existing deep-learning-based methods have demonstrated promising results in specific testing conditions, they suffer from three limitations: (1) lack of generalization capability from synthetic training data to real turbulence data; (2) failure to scale, hence causing memory and speed challenges when extending the idea to a large number of frames; (3) lack of a fast and accurate simulator to generate data for training neural networks. In this paper, we introduce the turbulence mitigation transformer (TMT) that explicitly addresses these issues. TMT brings three contributions: Firstly, TMT explicitly uses turbulence physics by decoupling the turbulence degradation and introducing a multi-scale loss for removing distortion, thus improving effectiveness. Secondly, TMT presents a new attention module along the temporal axis to extract extra features efficiently, thus improving memory and speed. Thirdly, TMT introduces a new simulator based on the Fourier sampler, temporal correlation, and flexible kernel size, thus improving our capability to synthesize better training data. TMT outperforms state-of-the-art video restoration models, especially in generalizing from synthetic to real turbulence data. Code, videos, and datasets are available at hrefhttps://xg416.github.io/TMThttps://xg416.github.io/TMT.