# Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views

arXiv: https://arxiv.org/abs/2307.00821 · HTML: https://mlsanity.com/paper/2307.00821/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 11 of 11)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [Optical Flow Estimation for Spiking Camera](https://mlsanity.com/paper/2110.03916/) (2021)
- [Spike Camera and Its Coding Methods](https://mlsanity.com/paper/2104.04669/) (2021)
- [Event-based Synthetic Aperture Imaging with a Hybrid Network](https://mlsanity.com/paper/2103.02376/) (2021)
- [An Efficient Coding Method for Spike Camera using Inter-Spike Intervals](https://mlsanity.com/paper/1912.09669/) (2019)
- [DeOccNet: Learning to See Through Foreground Occlusions in Light Fields](https://mlsanity.com/paper/1912.04459/) (2019)
- [Learning to See Through with Events](https://mlsanity.com/paper/2212.02219/) (2022)
- [I See-Through You: A Framework for Removing Foreground Occlusion in Both Sparse and Dense Light Field Images](https://mlsanity.com/paper/2301.06392/) (2023)

## Built on by (most cited first, 5 of 5)

- [SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams](https://mlsanity.com/paper/2403.09486/) (2024)
- [USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting](https://mlsanity.com/paper/2411.10504/) (2024)
- [Event Camera Guided Visual Media Restoration & 3D Reconstruction: A Survey](https://mlsanity.com/paper/2509.09971/) (2025)
- [SpikeMM: Flexi-Magnification of High-Speed Micro-Motions](https://mlsanity.com/paper/2406.00383/) (2024)
- [SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams](https://mlsanity.com/paper/2505.19487/) (2025)

## Abstract (arXiv)

The extraction of a clean background image by removing foreground occlusion holds immense practical significance, but it also presents several challenges. Presently, the majority of de-occlusion research focuses on addressing this issue through the extraction and synthesis of discrete images from calibrated camera arrays. Nonetheless, the restoration quality tends to suffer when faced with dense occlusions or high-speed motions due to limited perspectives and motion blur. To successfully remove dense foreground occlusion, an effective multi-view visual information integration approach is required. Introducing the spike camera as a novel type of neuromorphic sensor offers promising capabilities with its ultra-high temporal resolution and high dynamic range. In this paper, we propose an innovative solution for tackling the de-occlusion problem through continuous multi-view imaging using only one spike camera without any prior knowledge of camera intrinsic parameters and camera poses. By rapidly moving the spike camera, we continually capture the dense stream of spikes from the occluded scene. To process the spikes, we build a novel model SpkOccNet, in which we integrate information of spikes from continuous viewpoints within multi-windows, and propose a novel cross-view mutual attention mechanism for effective fusion and refinement. In addition, we contribute the first real-world spike-based dataset S-OCC for occlusion removal. The experimental results demonstrate that our proposed model efficiently removes dense occlusions in diverse scenes while exhibiting strong generalization.