Towards Real-time Video Compressive Sensing on Mobile Devices
Not on the map. Why?
Abstract (arXiv)
Video Snapshot Compressive Imaging (SCI) uses a low-speed 2D camera to capture high-speed scenes as snapshot compressed measurements, followed by a reconstruction algorithm to retrieve the high-speed video frames. The fast evolving mobile devices and existing high-performance video SCI reconstruction algorithms motivate us to develop mobile reconstruction methods for real-world applications. Yet, it is still challenging to deploy previous reconstruction algorithms on mobile devices due to the complex inference process, let alone real-time mobile reconstruction. To the best of our knowledge, there is no video SCI reconstruction model designed to run on the mobile devices. Towards this end, in this paper, we present an effective approach for video SCI reconstruction, dubbed MobileSCI, which can run at real-time speed on the mobile devices for the first time. Specifically, we first build a U-shaped 2D convolution-based architecture, which is much more efficient and mobile-friendly than previous state-of-the-art reconstruction methods. Besides, an efficient feature mixing block, based on the channel splitting and shuffling mechanisms, is introduced as a novel bottleneck block of our proposed MobileSCI to alleviate the computational burden. Finally, a customized knowledge distillation strategy is utilized to further improve the reconstruction quality. Extensive results on both simulated and real data show that our proposed MobileSCI can achieve superior reconstruction quality with high efficiency on the mobile devices. Particularly, we can reconstruct a 256 X 256 X 8 snapshot compressed measurement with real-time performance (about 35 FPS) on an iPhone 15. Code is available at https://github.com/mcao92/MobileSCI.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 14 Aug 2024.
Builds on
20 of 42 references.
References · 20
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2019Searching for MobileNetV3cited by 1,984
- 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
- 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
- 2019GhostNet: More Features from Cheap Operationscited by 410
- 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
- 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
- 2018Rank Minimization for Snapshot Compressive Imagingcited by 99
8 more
- 2020Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imagingcited by 78
- 2022Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstructioncited by 67
- 2022GhostNetV2: Enhance Cheap Operation with Long-Range Attentioncited by 44
- 2021Plug-and-Play Algorithms for Video Snapshot Compressive Imagingcited by 42
- 2021Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imagingcited by 31
- 2021MetaSCI: Scalable and Adaptive Reconstruction for Video Compressive Sensingcited by 29
- 2022Spatial-Temporal Transformer for Video Snapshot Compressive Imagingcited by 21
- 2024A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imagingcited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.