# Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streams

arXiv: https://arxiv.org/abs/2401.10461 · HTML: https://mlsanity.com/paper/2401.10461/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 20 of 20)

- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement](https://mlsanity.com/paper/2303.06705/) (2023)
- [Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation](https://mlsanity.com/paper/2103.00860/) (2021)
- [Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method](https://mlsanity.com/paper/2212.11548/) (2022)
- [Low-Light Image and Video Enhancement Using Deep Learning: A Survey](https://mlsanity.com/paper/2104.10729/) (2021)
- [Low-Light Image Enhancement via Structure Modeling and Guidance](https://mlsanity.com/paper/2305.05839/) (2023)
- [Learning Semantic-Aware Knowledge Guidance for Low-Light Image Enhancement](https://mlsanity.com/paper/2304.07039/) (2023)
- [A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement](https://mlsanity.com/paper/1711.00591/) (2017)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [Optical Flow Estimation for Spiking Camera](https://mlsanity.com/paper/2110.03916/) (2021)
- [1000x Faster Camera and Machine Vision with Ordinary Devices](https://mlsanity.com/paper/2201.09302/) (2022)
- [ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networks](https://mlsanity.com/paper/2306.03693/) (2023)
- [Optimal biasing and physical limits of DVS event noise](https://mlsanity.com/paper/2304.04019/) (2023)
- [Enhancing Adaptive History Reserving by Spiking Convolutional Block Attention Module in Recurrent Neural Networks](https://mlsanity.com/paper/2401.03719/) (2024)
- [Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer](https://mlsanity.com/paper/2208.12527/) (2022)
- [Recurrent Spike-based Image Restoration under General Illumination](https://mlsanity.com/paper/2308.03018/) (2023)
- [Uncertainty Guided Depth Fusion for Spike Camera](https://mlsanity.com/paper/2208.12653/) (2022)

## Abstract (arXiv)

As a neuromorphic sensor with high temporal resolution, spike camera can generate continuous binary spike streams to capture per-pixel light intensity. We can use reconstruction methods to restore scene details in high-speed scenarios. However, due to limited information in spike streams, low-light scenes are difficult to effectively reconstruct. In this paper, we propose a bidirectional recurrent-based reconstruction framework, including a Light-Robust Representation (LR-Rep) and a fusion module, to better handle such extreme conditions. LR-Rep is designed to aggregate temporal information in spike streams, and a fusion module is utilized to extract temporal features. Additionally, we have developed a reconstruction benchmark for high-speed low-light scenes. Light sources in the scenes are carefully aligned to real-world conditions. Experimental results demonstrate the superiority of our method, which also generalizes well to real spike streams. Related codes and proposed datasets will be released after publication.