# Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence

arXiv: https://arxiv.org/abs/2404.13605 · HTML: https://mlsanity.com/paper/2404.13605/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 21 Apr 2024.

## Builds on (most cited first, 19 of 19)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow](https://mlsanity.com/paper/2203.16896/) (2022)
- [Deformable Sprites for Unsupervised Video Decomposition](https://mlsanity.com/paper/2204.07151/) (2022)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model](https://mlsanity.com/paper/2207.10040/) (2022)
- [Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2209.03138/) (2022)
- [Imaging through the Atmosphere using Turbulence Mitigation Transformer](https://mlsanity.com/paper/2207.06465/) (2022)
- [Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulence](https://mlsanity.com/paper/2009.00071/) (2020)
- [Simulating Anisoplanatic Turbulence by Sampling Inter-modal and Spatially Correlated Zernike Coefficients](https://mlsanity.com/paper/2004.11210/) (2020)
- [Learning to restore images degraded by atmospheric turbulence using uncertainty](https://mlsanity.com/paper/2207.03447/) (2022)
- [Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Network](https://mlsanity.com/paper/2204.06989/) (2022)
- [Restoration of Atmospheric Turbulence-distorted Images via RPCA and Quasiconformal Maps](https://mlsanity.com/paper/1704.03140/) (2017)
- [Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model](https://mlsanity.com/paper/2207.06377/) (2022)
- [Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence](https://mlsanity.com/paper/2210.06713/) (2022)
- [DS-Net: Dynamic Spatiotemporal Network for Video Salient Object Detection](https://mlsanity.com/paper/2012.04886/) (2020)

## Built on by (most cited first, 7 of 7)

- [Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation](https://mlsanity.com/paper/2504.02697/) (2025)
- [MAMAT: 3D Mamba-Based Atmospheric Turbulence Removal and its Object Detection Capability](https://mlsanity.com/paper/2503.17700/) (2025)
- [EGTM: Event-guided Efficient Turbulence Mitigation](https://mlsanity.com/paper/2509.03808/) (2025)
- [RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator](https://mlsanity.com/paper/2508.11409/) (2025)
- [HATIR: Heat-Aware Diffusion for Turbulent Infrared Video Super-Resolution](https://mlsanity.com/paper/2601.04682/) (2026)
- [High-Quality and Efficient Turbulence Mitigation with Events](https://mlsanity.com/paper/2603.20708/) (2026)
- [ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes](https://mlsanity.com/paper/2605.21440/) (2026)

## Abstract (arXiv)

Tackling image degradation due to atmospheric turbulence, particularly in dynamic environment, remains a challenge for long-range imaging systems. Existing techniques have been primarily designed for static scenes or scenes with small motion. This paper presents the first segment-then-restore pipeline for restoring the videos of dynamic scenes in turbulent environment. We leverage mean optical flow with an unsupervised motion segmentation method to separate dynamic and static scene components prior to restoration. After camera shake compensation and segmentation, we introduce foreground/background enhancement leveraging the statistics of turbulence strength and a transformer model trained on a novel noise-based procedural turbulence generator for fast dataset augmentation. Benchmarked against existing restoration methods, our approach restores most of the geometric distortion and enhances sharpness for videos. We make our code, simulator, and data publicly available to advance the field of video restoration from turbulence: riponcs.github.io/TurbSegRes