# SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams

arXiv: https://arxiv.org/abs/2403.09486 · HTML: https://mlsanity.com/paper/2403.09486/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [Unsupervised Degradation Representation Learning for Blind Super-Resolution](https://mlsanity.com/paper/2104.00416/) (2021)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [Event-Based Fusion for Motion Deblurring with Cross-modal Attention](https://mlsanity.com/paper/2112.00167/) (2021)
- [Deep Image Deblurring: A Survey](https://mlsanity.com/paper/2201.10700/) (2022)
- [AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network](https://mlsanity.com/paper/2203.11799/) (2022)
- [Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots](https://mlsanity.com/paper/2203.06967/) (2022)
- [Unifying Motion Deblurring and Frame Interpolation with Events](https://mlsanity.com/paper/2203.12178/) (2022)
- [Learning to Extract a Video Sequence from a Single Motion-Blurred Image](https://mlsanity.com/paper/1804.04065/) (2018)
- [Motion Deblurring with Real Events](https://mlsanity.com/paper/2109.13695/) (2021)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [Exposure Trajectory Recovery from Motion Blur](https://mlsanity.com/paper/2010.02484/) (2020)
- [A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild](https://mlsanity.com/paper/2301.05191/) (2023)
- [1000x Faster Camera and Machine Vision with Ordinary Devices](https://mlsanity.com/paper/2201.09302/) (2022)
- [E-CIR: Event-Enhanced Continuous Intensity Recovery](https://mlsanity.com/paper/2203.01935/) (2022)
- [Blur Interpolation Transformer for Real-World Motion from Blur](https://mlsanity.com/paper/2211.11423/) (2022)
- [FBI-Denoiser: Fast Blind Image Denoiser for Poisson-Gaussian Noise](https://mlsanity.com/paper/2105.10967/) (2021)
- [Generalizing Event-Based Motion Deblurring in Real-World Scenarios](https://mlsanity.com/paper/2308.05932/) (2023)
- [SpikeCV: Open a Continuous Computer Vision Era](https://mlsanity.com/paper/2303.11684/) (2023)
- [Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks](https://mlsanity.com/paper/1904.13007/) (2019)
- [Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views](https://mlsanity.com/paper/2307.00821/) (2023)

## Built on by (most cited first, 6 of 6)

- [Differential Coding for Training-Free ANN-to-SNN Conversion](https://mlsanity.com/paper/2503.00301/) (2025)
- [USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting](https://mlsanity.com/paper/2411.10504/) (2024)
- [SpikeGS: Learning 3D Gaussian Fields from Continuous Spike Stream](https://mlsanity.com/paper/2409.15176/) (2024)
- [SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams](https://mlsanity.com/paper/2503.20315/) (2025)
- [SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams](https://mlsanity.com/paper/2505.19487/) (2025)
- [SpikeGen: Decoupled "Rods and Cones" Visual Representation Processing with Latent Generative Framework](https://mlsanity.com/paper/2505.18049/) (2025)

## Abstract (arXiv)

Reconstructing a sequence of sharp images from the blurry input is crucial for enhancing our insights into the captured scene and poses a significant challenge due to the limited temporal features embedded in the image. Spike cameras, sampling at rates up to 40,000 Hz, have proven effective in capturing motion features and beneficial for solving this ill-posed problem. Nonetheless, existing methods fall into the supervised learning paradigm, which suffers from notable performance degradation when applied to real-world scenarios that diverge from the synthetic training data domain. Moreover, the quality of reconstructed images is capped by the generated images based on motion analysis interpolation, which inherently differs from the actual scene, affecting the generalization ability of these methods in real high-speed scenarios. To address these challenges, we propose the first self-supervised framework for the task of spike-guided motion deblurring. Our approach begins with the formulation of a spike-guided deblurring model that explores the theoretical relationships among spike streams, blurry images, and their corresponding sharp sequences. We subsequently develop a self-supervised cascaded framework to alleviate the issues of spike noise and spatial-resolution mismatching encountered in the deblurring model. With knowledge distillation and re-blurring loss, we further design a lightweight deblur network to generate high-quality sequences with brightness and texture consistency with the original input. Quantitative and qualitative experiments conducted on our real-world and synthetic datasets with spikes validate the superior generalization of the proposed framework. Our code, data and trained models will be available at urlhttps://github.com/chenkang455/S-SDM.