# Modulo Video Recovery via Selective Spatiotemporal Vision Transformer

arXiv: https://arxiv.org/abs/2511.07479 · HTML: https://mlsanity.com/paper/2511.07479/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 46 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 22 of 22)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Improving Adversarial Transferability via Neuron Attribution-Based Attacks](https://mlsanity.com/paper/2204.00008/) (2022)
- [Temporally Distributed Networks for Fast Video Semantic Segmentation](https://mlsanity.com/paper/2004.01800/) (2020)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization](https://mlsanity.com/paper/2303.15754/) (2023)
- [NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results](https://mlsanity.com/paper/2106.01439/) (2021)
- [Efficient Segmentation: Learning Downsampling Near Semantic Boundaries](https://mlsanity.com/paper/1907.07156/) (2019)
- [On Unlimited Sampling and Reconstruction](https://mlsanity.com/paper/1905.03901/) (2019)
- [Convolutional Sparse Coding for High Dynamic Range Imaging](https://mlsanity.com/paper/1806.04942/) (2018)
- [Unlimited Sampling from Theory to Practice: Fourier-Prony Recovery and Prototype ADC](https://mlsanity.com/paper/2105.05818/) (2021)
- [Deep Joint Demosaicing and High Dynamic Range Imaging within a Single Shot](https://mlsanity.com/paper/2111.07281/) (2021)
- [Generalized Approximate Message Passing for Unlimited Sampling of Sparse Signals](https://mlsanity.com/paper/1807.03182/) (2018)
- [Unsupervised HDR Image and Video Tone Mapping via Contrastive Learning](https://mlsanity.com/paper/2303.07327/) (2023)
- [Robust Multi-Image HDR Reconstruction for the Modulo Camera](https://mlsanity.com/paper/1707.01317/) (2017)
- [Signal Reconstruction from Modulo Observations](https://mlsanity.com/paper/1812.00557/) (2018)
- [Reconstruction from Periodic Nonlinearities, With Applications to HDR Imaging](https://mlsanity.com/paper/1710.00109/) (2017)
- [On denoising modulo 1 samples of a function](https://mlsanity.com/paper/1710.10210/) (2017)
- [High Dynamic Range Image Reconstruction via Deep Explicit Polynomial Curve Estimation](https://mlsanity.com/paper/2307.16426/) (2023)

## Abstract (arXiv)

Conventional image sensors have limited dynamic range, causing saturation in high-dynamic-range (HDR) scenes. Modulo cameras address this by folding incident irradiance into a bounded range, yet require specialized unwrapping algorithms to reconstruct the underlying signal. Unlike HDR recovery, which extends dynamic range from conventional sampling, modulo recovery restores actual values from folded samples. Despite being introduced over a decade ago, progress in modulo image recovery has been slow, especially in the use of modern deep learning techniques. In this work, we demonstrate that standard HDR methods are unsuitable for modulo recovery. Transformers, however, can capture global dependencies and spatial-temporal relationships crucial for resolving folded video frames. Still, adapting existing Transformer architectures for modulo recovery demands novel techniques. To this end, we present Selective Spatiotemporal Vision Transformer (SSViT), the first deep learning framework for modulo video reconstruction. SSViT employs a token selection strategy to improve efficiency and concentrate on the most critical regions. Experiments confirm that SSViT produces high-quality reconstructions from 8-bit folded videos and achieves state-of-the-art performance in modulo video recovery.