# EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting

arXiv: https://arxiv.org/abs/2405.14959 · HTML: https://mlsanity.com/paper/2405.14959/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 30 of 35)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Neural Rays for Occlusion-aware Image-based Rendering](https://mlsanity.com/paper/2107.13421/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing](https://mlsanity.com/paper/1911.13225/) (2019)
- [GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis](https://mlsanity.com/paper/2312.02155/) (2023)
- [Efficient Neural Radiance Fields for Interactive Free-viewpoint Video](https://mlsanity.com/paper/2112.01517/) (2021)
- [NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing](https://mlsanity.com/paper/2207.11911/) (2022)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Learning Monocular Dense Depth from Events](https://mlsanity.com/paper/2010.08350/) (2020)
- [E-NeRF: Neural Radiance Fields from a Moving Event Camera](https://mlsanity.com/paper/2208.11300/) (2022)
- [Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection](https://mlsanity.com/paper/2303.10840/) (2023)
- [Event Enhanced High-Quality Image Recovery](https://mlsanity.com/paper/2007.08336/) (2020)
- [Semi-Dense 3D Reconstruction with a Stereo Event Camera](https://mlsanity.com/paper/1807.07429/) (2018)
- [NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions](https://mlsanity.com/paper/2309.15426/) (2023)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Ev-NeRF: Event Based Neural Radiance Field](https://mlsanity.com/paper/2206.12455/) (2022)
- [Ponder: Point Cloud Pre-training via Neural Rendering](https://mlsanity.com/paper/2301.00157/) (2022)

## Built on by (most cited first, 20 of 20)

- [EvaGaussians: Event Stream Assisted Gaussian Splatting from Blurry Images](https://mlsanity.com/paper/2405.20224/) (2024)
- [IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera](https://mlsanity.com/paper/2410.08107/) (2024)
- [EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering](https://mlsanity.com/paper/2412.07293/) (2024)
- [SweepEvGS: Event-Based 3D Gaussian Splatting for Macro and Micro Radiance Field Rendering from a Single Sweep](https://mlsanity.com/paper/2412.11579/) (2024)
- [E-4DGS: High-Fidelity Dynamic Reconstruction from the Multi-view Event Cameras](https://mlsanity.com/paper/2508.09912/) (2025)
- [GS2E: Gaussian Splatting is an Effective Data Generator for Event Stream Generation](https://mlsanity.com/paper/2505.15287/) (2025)
- [SpikeGS: Reconstruct 3D scene via fast-moving bio-inspired sensors](https://mlsanity.com/paper/2407.03771/) (2024)
- [A Survey of 3D Reconstruction with Event Cameras](https://mlsanity.com/paper/2505.08438/) (2025)
- [E-3DGS: Gaussian Splatting with Exposure and Motion Events](https://mlsanity.com/paper/2410.16995/) (2024)
- [USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting](https://mlsanity.com/paper/2411.10504/) (2024)
- [DEGS: Deformable Event-based 3D Gaussian Splatting from RGB and Event Stream](https://mlsanity.com/paper/2510.07752/) (2025)
- [E2EGS: Event-to-Edge Gaussian Splatting for Pose-Free 3D Reconstruction](https://mlsanity.com/paper/2603.14684/) (2026)
- [SpikeGS: Learning 3D Gaussian Fields from Continuous Spike Stream](https://mlsanity.com/paper/2409.15176/) (2024)
- [STD-GS: Exploring Frame-Event Interaction for SpatioTemporal-Disentangled Gaussian Splatting to Reconstruct High-Dynamic Scene](https://mlsanity.com/paper/2506.23157/) (2025)
- [Event-Aided Sharp Radiance Field Reconstruction for Fast-Flying Drones](https://mlsanity.com/paper/2602.21101/) (2026)
- [Learning Structural Latent Points for Efficient Visual Representations in Robotic Manipulation](https://mlsanity.com/paper/2605.21258/) (2026)
- [From Events to Enhancement: A Survey on Event-Based Imaging Technologies](https://mlsanity.com/paper/2505.05488/) (2025)
- [Event3R: Asynchronous-to-Global 3D Reconstruction from Event Camera via Spatial-Temporal Feature Aggregation](https://mlsanity.com/paper/2607.15727/) (2026)
- [ERF-GS: Reconstructing Fast Motion from Disjoint Event-RGB Viewpoints](https://mlsanity.com/paper/2608.08531/) (2026)
- [EvTrajGS: Accurate and Efficient 3D Gaussian Splatting from Unposed Event Streams](https://mlsanity.com/paper/2608.08585/) (2026)

## Abstract (arXiv)

Event cameras offer promising advantages such as high dynamic range and low latency, making them well-suited for challenging lighting conditions and fast-moving scenarios. However, reconstructing 3D scenes from raw event streams is difficult because event data is sparse and does not carry absolute color information. To release its potential in 3D reconstruction, we propose the first event-based generalizable 3D reconstruction framework, called EvGGS, which reconstructs scenes as 3D Gaussians from only event input in a feedforward manner and can generalize to unseen cases without any retraining. This framework includes a depth estimation module, an intensity reconstruction module, and a Gaussian regression module. These submodules connect in a cascading manner, and we collaboratively train them with a designed joint loss to make them mutually promote. To facilitate related studies, we build a novel event-based 3D dataset with various material objects and calibrated labels of grayscale images, depth maps, camera poses, and silhouettes. Experiments show models that have jointly trained significantly outperform those trained individually. Our approach performs better than all baselines in reconstruction quality, and depth/intensity predictions with satisfactory rendering speed.