# DeTurb: Atmospheric Turbulence Mitigation with Deformable 3D Convolutions and 3D Swin Transformers

arXiv: https://arxiv.org/abs/2407.20855 · HTML: https://mlsanity.com/paper/2407.20855/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Parallel Diffusion Models of Operator and Image for Blind Inverse Problems](https://mlsanity.com/paper/2211.10656/) (2022)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model](https://mlsanity.com/paper/2207.10040/) (2022)
- [AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2208.11284/) (2022)
- [Deformable 3D Convolution for Video Super-Resolution](https://mlsanity.com/paper/2004.02803/) (2020)
- [Imaging through the Atmosphere using Turbulence Mitigation Transformer](https://mlsanity.com/paper/2207.06465/) (2022)
- [Physics-Driven Turbulence Image Restoration with Stochastic Refinement](https://mlsanity.com/paper/2307.10603/) (2023)
- [Learning to restore images degraded by atmospheric turbulence using uncertainty](https://mlsanity.com/paper/2207.03447/) (2022)
- [VDPVE: VQA Dataset for Perceptual Video Enhancement](https://mlsanity.com/paper/2303.09290/) (2023)
- [Spatio-Temporal Turbulence Mitigation: A Translational Perspective](https://mlsanity.com/paper/2401.04244/) (2024)
- [LTT-GAN: Looking Through Turbulence by Inverting GANs](https://mlsanity.com/paper/2112.02379/) (2021)
- [Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Network](https://mlsanity.com/paper/2204.06989/) (2022)
- [Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time](https://mlsanity.com/paper/2303.15043/) (2023)
- [Atmospheric turbulence removal using convolutional neural network](https://mlsanity.com/paper/1912.11350/) (2019)
- [A Spatio-temporal Aligned SUNet Model for Low-light Video Enhancement](https://mlsanity.com/paper/2403.02408/) (2024)
- [BVI-RLV: A Fully Registered Dataset for Low-Light Video Enhancement](https://mlsanity.com/paper/2407.03535/) (2024)
- [Atmospheric Turbulence Correction via Variational Deep Diffusion](https://mlsanity.com/paper/2305.05077/) (2023)
- [Object recognition in atmospheric turbulence scenes](https://mlsanity.com/paper/2210.14318/) (2022)

## Built on by (most cited first, 6 of 6)

- [MAMAT: 3D Mamba-Based Atmospheric Turbulence Removal and its Object Detection Capability](https://mlsanity.com/paper/2503.17700/) (2025)
- [DMAT: An End-to-End Framework for Joint Atmospheric Turbulence Mitigation and Object Detection](https://mlsanity.com/paper/2507.04323/) (2025)
- [RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator](https://mlsanity.com/paper/2508.11409/) (2025)
- [GSTurb: Gaussian Splatting for Atmospheric Turbulence Mitigation](https://mlsanity.com/paper/2602.22800/) (2026)
- [High-Quality and Efficient Turbulence Mitigation with Events](https://mlsanity.com/paper/2603.20708/) (2026)
- [ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes](https://mlsanity.com/paper/2605.21440/) (2026)

## Abstract (arXiv)

Atmospheric turbulence in long-range imaging significantly degrades the quality and fidelity of captured scenes due to random variations in both spatial and temporal dimensions. These distortions present a formidable challenge across various applications, from surveillance to astronomy, necessitating robust mitigation strategies. While model-based approaches achieve good results, they are very slow. Deep learning approaches show promise in image and video restoration but have struggled to address these spatiotemporal variant distortions effectively. This paper proposes a new framework that combines geometric restoration with an enhancement module. Random perturbations and geometric distortion are removed using a pyramid architecture with deformable 3D convolutions, resulting in aligned frames. These frames are then used to reconstruct a sharp, clear image via a multi-scale architecture of 3D Swin Transformers. The proposed framework demonstrates superior performance over the state of the art for both synthetic and real atmospheric turbulence effects, with reasonable speed and model size.