# SpikeGS: 3D Gaussian Splatting from Spike Streams with High-Speed Camera Motion

arXiv: https://arxiv.org/abs/2407.10062 · HTML: https://mlsanity.com/paper/2407.10062/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 14 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [NeRF-SR: High-Quality Neural Radiance Fields using Supersampling](https://mlsanity.com/paper/2112.01759/) (2021)
- [AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network](https://mlsanity.com/paper/2203.11799/) (2022)
- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Unpaired Learning of Deep Image Denoising](https://mlsanity.com/paper/2008.13711/) (2020)
- [E-NeRF: Neural Radiance Fields from a Moving Event Camera](https://mlsanity.com/paper/2208.11300/) (2022)
- [BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting](https://mlsanity.com/paper/2403.11831/) (2024)
- [BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling](https://mlsanity.com/paper/2403.04926/) (2024)
- [LG-BPN: Local and Global Blind-Patch Network for Self-Supervised Real-World Denoising](https://mlsanity.com/paper/2304.00534/) (2023)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising](https://mlsanity.com/paper/2303.14934/) (2023)
- [Deformable Neural Radiance Fields using RGB and Event Cameras](https://mlsanity.com/paper/2309.08416/) (2023)
- [Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion](https://mlsanity.com/paper/2403.13327/) (2024)
- [1000x Faster Camera and Machine Vision with Ordinary Devices](https://mlsanity.com/paper/2201.09302/) (2022)
- [PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis](https://mlsanity.com/paper/2202.04879/) (2022)

## Built on by (most cited first, 6 of 6)

- [SOTA: Spike-Navigated Optimal TrAnsport Saliency Region Detection in Composite-bias Videos](https://mlsanity.com/paper/2505.00394/) (2025)
- [USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting](https://mlsanity.com/paper/2411.10504/) (2024)
- [SpikeGS: Learning 3D Gaussian Fields from Continuous Spike Stream](https://mlsanity.com/paper/2409.15176/) (2024)
- [SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams](https://mlsanity.com/paper/2505.19487/) (2025)
- [FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Vision](https://mlsanity.com/paper/2603.19770/) (2026)
- [Robust 3DGS-based SLAM via Adaptive Kernel Smoothing](https://mlsanity.com/paper/2511.23221/) (2025)

## Abstract (arXiv)

Novel View Synthesis plays a crucial role by generating new 2D renderings from multi-view images of 3D scenes. However, capturing high-speed scenes with conventional cameras often leads to motion blur, hindering the effectiveness of 3D reconstruction. To address this challenge, high-frame-rate dense 3D reconstruction emerges as a vital technique, enabling detailed and accurate modeling of real-world objects or scenes in various fields, including Virtual Reality or embodied AI. Spike cameras, a novel type of neuromorphic sensor, continuously record scenes with an ultra-high temporal resolution, showing potential for accurate 3D reconstruction. Despite their promise, existing approaches, such as applying Neural Radiance Fields (NeRF) to spike cameras, encounter challenges due to the time-consuming rendering process. To address this issue, we make the first attempt to introduce the 3D Gaussian Splatting (3DGS) into spike cameras in high-speed capture, providing 3DGS as dense and continuous clues of views, then constructing SpikeGS. Specifically, to train SpikeGS, we establish computational equations between the rendering process of 3DGS and the processes of instantaneous imaging and exposing-like imaging of the continuous spike stream. Besides, we build a very lightweight but effective mapping process from spikes to instant images to support training. Furthermore, we introduced a new spike-based 3D rendering dataset for validation. Extensive experiments have demonstrated our method possesses the high quality of novel view rendering, proving the tremendous potential of spike cameras in modeling 3D scenes.