mlsanityData to 7 Oct 2026

ConVRT: Consistent Video Restoration Through Turbulence with Test-time Optimization of Neural Video Representations

arXiv 2312.04679 ↗First posted 7 Dec 2023eess.IV cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

tmospheric turbulence presents a significant challenge in long-range imaging. Current restoration algorithms often struggle with temporal inconsistency, as well as limited generalization ability across varying turbulence levels and scene content different than the training data. To tackle these issues, we introduce a self-supervised method, Consistent Video Restoration through Turbulence (ConVRT) a test-time optimization method featuring a neural video representation designed to enhance temporal consistency in restoration. A key innovation of ConVRT is the integration of a pretrained vision-language model (CLIP) for semantic-oriented supervision, which steers the restoration towards sharp, photorealistic images in the CLIP latent space. We further develop a principled selection strategy of text prompts, based on their statistical correlation with a perceptual metric. ConVRT's test-time optimization allows it to adapt to a wide range of real-world turbulence conditions, effectively leveraging the insights gained from pre-trained models on simulated data. ConVRT offers a comprehensive and effective solution for mitigating real-world turbulence in dynamic videos.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

38 of 57 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  6. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  9. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  10. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  11. 2021NeRV: Neural Representations for Videoscited by 255
  12. 2018Learning Blind Video Temporal Consistencycited by 247
26 more
  1. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  2. 2023Tracking Everything Everywhere All at Oncecited by 190
  3. 2023HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Samplingcited by 173
  4. 2021Modulated Periodic Activations for Generalizable Local Functional Representationscited by 140
  5. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  6. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  7. 2023HNeRV: A Hybrid Neural Representation for Videoscited by 100
  8. 2022Deformable Sprites for Unsupervised Video Decompositioncited by 61
  9. 2016Artistic style transfer for videoscited by 54
  10. 2023Shape-aware Text-driven Layered Video Editingcited by 54
  11. 2021Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transformcited by 50
  12. 2019Video Object Segmentation and Tracking: A Surveycited by 49
  13. 2022Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Modelcited by 37
  14. 2022Imaging through the Atmosphere using Turbulence Mitigation Transformercited by 31
  15. 2020Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulencecited by 30
  16. 2020Deep Phase Decoder: Self-calibrating phase microscopy with an untrained deep neural networkcited by 24
  17. 2022Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Networkcited by 21
  18. 2018Atmospheric turbulence mitigation for sequences with moving objects using recursive image fusioncited by 17
  19. 2022PRIF: Primary Ray-based Implicit Functioncited by 17
  20. 2022TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulencecited by 11
  21. 20233D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fieldscited by 7
  22. 2023Scattering and Gathering for Spatially Varying Blurscited by 6
  23. 2023Learning Lens Blur Fieldscited by 6
  24. 2022VIINTER: View Interpolation with Implicit Neural Representations of Imagescited by 5
  25. 2023FPM-INR: Fourier ptychographic microscopy image stack reconstruction using implicit neural representationscited by 4
  26. 2023Local Conditional Neural Fields for Versatile and Generalizable Large-Scale Reconstructions in Computational Imagingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation
  2. 2026FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity
  3. 2025Temporal-Consistent Video Restoration with Pre-trained Diffusion Models
  4. 2026D2-VR: Degradation-Robust and Distilled Video Restoration with Synergistic Optimization Strategy
  5. 2026FastVR: Efficient Streaming Video Restoration with One-Step Diffusion
  6. 2025SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration
  7. 2025SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training
  8. 2025LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration