# SpikeNVS: Enhancing Novel View Synthesis from Blurry Images via Spike Camera

arXiv: https://arxiv.org/abs/2404.06710 · HTML: https://mlsanity.com/paper/2404.06710/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 10 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Deblur-NeRF: Neural Radiance Fields from Blurry Images](https://mlsanity.com/paper/2111.14292/) (2021)
- [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [EfficientNeRF: Efficient Neural Radiance Fields](https://mlsanity.com/paper/2206.00878/) (2022)
- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Ev-NeRF: Event Based Neural Radiance Field](https://mlsanity.com/paper/2206.12455/) (2022)
- [Multi-Bracket High Dynamic Range Imaging with Event Cameras](https://mlsanity.com/paper/2203.06622/) (2022)
- [Learning to Super Resolve Intensity Images from Events](https://mlsanity.com/paper/1912.01196/) (2019)
- [E-CIR: Event-Enhanced Continuous Intensity Recovery](https://mlsanity.com/paper/2203.01935/) (2022)
- [Spike Camera and Its Coding Methods](https://mlsanity.com/paper/2104.04669/) (2021)

## Built on by (most cited first, 4 of 4)

- [USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting](https://mlsanity.com/paper/2411.10504/) (2024)
- [SpikeGS: Learning 3D Gaussian Fields from Continuous Spike Stream](https://mlsanity.com/paper/2409.15176/) (2024)
- [SpikeGen: Decoupled "Rods and Cones" Visual Representation Processing with Latent Generative Framework](https://mlsanity.com/paper/2505.18049/) (2025)
- [SPACT18: Spiking Human Action Recognition Benchmark Dataset with Complementary RGB and Thermal Modalities](https://mlsanity.com/paper/2507.16151/) (2025)

## Abstract (arXiv)

One of the most critical factors in achieving sharp Novel View Synthesis (NVS) using neural field methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) is the quality of the training images. However, Conventional RGB cameras are susceptible to motion blur. In contrast, neuromorphic cameras like event and spike cameras inherently capture more comprehensive temporal information, which can provide a sharp representation of the scene as additional training data. Recent methods have explored the integration of event cameras to improve the quality of NVS. The event-RGB approaches have some limitations, such as high training costs and the inability to work effectively in the background. Instead, our study introduces a new method that uses the spike camera to overcome these limitations. By considering texture reconstruction from spike streams as ground truth, we design the Texture from Spike (TfS) loss. Since the spike camera relies on temporal integration instead of temporal differentiation used by event cameras, our proposed TfS loss maintains manageable training costs. It handles foreground objects with backgrounds simultaneously. We also provide a real-world dataset captured with our spike-RGB camera system to facilitate future research endeavors. We conduct extensive experiments using synthetic and real-world datasets to demonstrate that our design can enhance novel view synthesis across NeRF and 3DGS. The code and dataset will be made available for public access.