# NeRT: Implicit Neural Representations for General Unsupervised Turbulence Mitigation

arXiv: https://arxiv.org/abs/2308.00622 · HTML: https://mlsanity.com/paper/2308.00622/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 1 Aug 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [Multi-Temporal Recurrent Neural Networks For Progressive Non-Uniform Single Image Deblurring With Incremental Temporal Training](https://mlsanity.com/paper/1911.07410/) (2019)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model](https://mlsanity.com/paper/2207.10040/) (2022)
- [AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2208.11284/) (2022)
- [Imaging through the Atmosphere using Turbulence Mitigation Transformer](https://mlsanity.com/paper/2207.06465/) (2022)
- [Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulence](https://mlsanity.com/paper/2009.00071/) (2020)
- [Simulating Anisoplanatic Turbulence by Sampling Inter-modal and Spatially Correlated Zernike Coefficients](https://mlsanity.com/paper/2004.11210/) (2020)
- [Learning to restore images degraded by atmospheric turbulence using uncertainty](https://mlsanity.com/paper/2207.03447/) (2022)
- [LTT-GAN: Looking Through Turbulence by Inverting GANs](https://mlsanity.com/paper/2112.02379/) (2021)
- [Atmospheric turbulence mitigation for sequences with moving objects using recursive image fusion](https://mlsanity.com/paper/1808.03550/) (2018)
- [Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model](https://mlsanity.com/paper/2207.06377/) (2022)
- [Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence](https://mlsanity.com/paper/2210.06713/) (2022)
- [FedAudio: A Federated Learning Benchmark for Audio Tasks](https://mlsanity.com/paper/2210.15707/) (2022)
- [Atmospheric turbulence removal using convolutional neural network](https://mlsanity.com/paper/1912.11350/) (2019)
- [TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulence](https://mlsanity.com/paper/2203.06764/) (2022)

## Built on by (most cited first, 2 of 2)

- [Spatio-Temporal Turbulence Mitigation: A Translational Perspective](https://mlsanity.com/paper/2401.04244/) (2024)
- [PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations](https://mlsanity.com/paper/2402.16913/) (2024)

## Abstract (arXiv)

The atmospheric and water turbulence mitigation problems have emerged as challenging inverse problems in computer vision and optics communities over the years. However, current methods either rely heavily on the quality of the training dataset or fail to generalize over various scenarios, such as static scenes, dynamic scenes, and text reconstructions. We propose a general implicit neural representation for unsupervised atmospheric and water turbulence mitigation (NeRT). NeRT leverages the implicit neural representations and the physically correct tilt-then-blur turbulence model to reconstruct the clean, undistorted image, given only dozens of distorted input images. Moreover, we show that NeRT outperforms the state-of-the-art through various qualitative and quantitative evaluations of atmospheric and water turbulence datasets. Furthermore, we demonstrate the ability of NeRT to eliminate uncontrolled turbulence from real-world environments. Lastly, we incorporate NeRT into continuously captured video sequences and demonstrate 48 × speedup.