mlsanityData to 14 Oct 2026

Recurrent Spike-based Image Restoration under General Illumination

arXiv 2308.03018 ↗First posted 6 Aug 2023cs.CV eess.IVcited by 5

Not on the map. Why?

Abstract (arXiv)

Spike camera is a new type of bio-inspired vision sensor that records light intensity in the form of a spike array with high temporal resolution (20,000 Hz). This new paradigm of vision sensor offers significant advantages for many vision tasks such as high speed image reconstruction. However, existing spike-based approaches typically assume that the scenes are with sufficient light intensity, which is usually unavailable in many real-world scenarios such as rainy days or dusk scenes. To unlock more spike-based application scenarios, we propose a Recurrent Spike-based Image Restoration (RSIR) network, which is the first work towards restoring clear images from spike arrays under general illumination. Specifically, to accurately describe the noise distribution under different illuminations, we build a physical-based spike noise model according to the sampling process of the spike camera. Based on the noise model, we design our RSIR network which consists of an adaptive spike transformation module, a recurrent temporal feature fusion module, and a frequency-based spike denoising module. Our RSIR can process the spike array in a recursive manner to ensure that the spike temporal information is well utilized. In the training process, we generate the simulated spike data based on our noise model to train our network. Extensive experiments on real-world datasets with different illuminations demonstrate the effectiveness of the proposed network. The code and dataset are released at https://github.com/BIT-Vision/RSIR.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 35 of its 57 references that are in the map. First posted on arXiv 6 Aug 2023.

Builds on

35 of 57 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2017Deformable Convolutional Networkscited by 1,488
  4. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,358
  5. 2017Deep Image Priorcited by 1,098
  6. 2019EDVR: Video Restoration with Enhanced Deformable Convolutional Networkscited by 494
  7. 2018Learning to See in the Darkcited by 460
  8. 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 364
  9. 2019Noise2Self: Blind Denoising by Self-Supervisioncited by 276
  10. 2018Toward Convolutional Blind Denoising of Real Photographscited by 274
  11. 2018Unprocessing Images for Learned Raw Denoisingcited by 250
  12. 2018TDAN: Temporally Deformable Alignment Network for Video Super-Resolutioncited by 226
23 more
  1. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 200
  2. 2019High-Quality Self-Supervised Deep Image Denoisingcited by 161
  3. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  4. 2017Burst Denoising with Kernel Prediction Networkscited by 157
  5. 2016Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensationcited by 156
  6. 2020A Physics-based Noise Formation Model for Extreme Low-light Raw Denoisingcited by 138
  7. 2019Noisier2Noise: Learning to Denoise from Unpaired Noisy Datacited by 126
  8. 2018Asynchronous, Photometric Feature Tracking using Events and Framescited by 118
  9. 2019Variational Denoising Network: Toward Blind Noise Modeling and Removalcited by 117
  10. 2019Noise Flow: Noise Modeling with Conditional Normalizing Flowscited by 108
  11. 2022AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Networkcited by 93
  12. 2019DeepRED: Deep Image Prior Powered by REDcited by 85
  13. 2021Rethinking Noise Synthesis and Modeling in Raw Denoisingcited by 62
  14. 2019DVDnet: A Fast Network for Deep Video Denoisingcited by 60
  15. 2022Dancing under the stars: video denoising in starlightcited by 47
  16. 2021Rethinking Deep Image Prior for Denoisingcited by 38
  17. 2021Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusioncited by 36
  18. 2021On Measuring and Controlling the Spectral Bias of the Deep Image Priorcited by 36
  19. 2019A Retina-inspired Sampling Method for Visual Texture Reconstructioncited by 34
  20. 2020Learning Camera-Aware Noise Modelscited by 32
  21. 2022Variational Deep Image Restorationcited by 24
  22. 2022Estimating Fine-Grained Noise Model via Contrastive Learningcited by 14
  23. 2020Unpaired Image Denoisingcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 1.

5 papers.

  1. 2024SpikeNeRF: Learning Neural Radiance Fields from Continuous Spike Streamcited by 7
  2. 2024Finding Visual Saliency in Continuous Spike Streamcited by 3
  3. 2024SpikeGS: Learning 3D Gaussian Fields from Continuous Spike Streamcited by 1
  4. 2025Rethinking High-speed Image Reconstruction Framework with Spike Cameracited by 1
  5. 2024Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streamscited by 0