# Sampling-Priors-Augmented Deep Unfolding Network for Robust Video Compressive Sensing

arXiv: https://arxiv.org/abs/2307.07291 · HTML: https://mlsanity.com/paper/2307.07291/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation](https://mlsanity.com/paper/1907.01361/) (2019)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Rank Minimization for Snapshot Compressive Imaging](https://mlsanity.com/paper/1807.07837/) (2018)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging](https://mlsanity.com/paper/2003.13654/) (2020)
- [AMP-Net: Denoising based Deep Unfolding for Compressive Image Sensing](https://mlsanity.com/paper/2004.10078/) (2020)
- [Coded aperture compressive temporal imaging](https://mlsanity.com/paper/1302.2575/) (2013)
- [GAP-net for Snapshot Compressive Imaging](https://mlsanity.com/paper/2012.08364/) (2020)
- [Plug-and-Play Algorithms for Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2101.04822/) (2021)
- [COAST: COntrollable Arbitrary-Sampling NeTwork for Compressive Sensing](https://mlsanity.com/paper/2107.07225/) (2021)
- [Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging](https://mlsanity.com/paper/2109.06548/) (2021)
- [Memory-Efficient Network for Large-scale Video Compressive Sensing](https://mlsanity.com/paper/2103.03089/) (2021)
- [Low-Cost Compressive Sensing for Color Video and Depth](https://mlsanity.com/paper/1402.6932/) (2014)

## Abstract (arXiv)

Video Compressed Sensing (VCS) aims to reconstruct multiple frames from one single captured measurement, thus achieving high-speed scene recording with a low-frame-rate sensor. Although there have been impressive advances in VCS recently, those state-of-the-art (SOTA) methods also significantly increase model complexity and suffer from poor generality and robustness, which means that those networks need to be retrained to accommodate the new system. Such limitations hinder the real-time imaging and practical deployment of models. In this work, we propose a Sampling-Priors-Augmented Deep Unfolding Network (SPA-DUN) for efficient and robust VCS reconstruction. Under the optimization-inspired deep unfolding framework, a lightweight and efficient U-net is exploited to downsize the model while improving overall performance. Moreover, the prior knowledge from the sampling model is utilized to dynamically modulate the network features to enable single SPA-DUN to handle arbitrary sampling settings, augmenting interpretability and generality. Extensive experiments on both simulation and real datasets demonstrate that SPA-DUN is not only applicable for various sampling settings with one single model but also achieves SOTA performance with incredible efficiency.