# Semantic-E2VID: a Semantic-Enriched Paradigm for Event-to-Video Reconstruction

arXiv: https://arxiv.org/abs/2510.17347 · HTML: https://mlsanity.com/paper/2510.17347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 54 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [Learning Blind Video Temporal Consistency](https://mlsanity.com/paper/1808.00449/) (2018)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.02618/) (2021)
- [HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks](https://mlsanity.com/paper/2305.06382/) (2023)
- [Semantic Pyramid for Image Generation](https://mlsanity.com/paper/2003.06221/) (2020)
- [Seeing Motion at Nighttime with an Event Camera](https://mlsanity.com/paper/2404.11884/) (2024)
- [Distilling Semantic Priors from SAM to Efficient Image Restoration Models](https://mlsanity.com/paper/2403.16368/) (2024)
- [EVREAL: Towards a Comprehensive Benchmark and Analysis Suite for Event-based Video Reconstruction](https://mlsanity.com/paper/2305.00434/) (2023)
- [An Empirical Study on the Robustness of the Segment Anything Model (SAM)](https://mlsanity.com/paper/2305.06422/) (2023)
- [Ev-GS: Event-based Gaussian splatting for Efficient and Accurate Radiance Field Rendering](https://mlsanity.com/paper/2407.11343/) (2024)
- [Segment Anything Model is a Good Teacher for Local Feature Learning](https://mlsanity.com/paper/2309.16992/) (2023)
- [Temporal Residual Guided Diffusion Framework for Event-Driven Video Reconstruction](https://mlsanity.com/paper/2407.10636/) (2024)
- [ContextSeg: Sketch Semantic Segmentation by Querying the Context with Attention](https://mlsanity.com/paper/2311.16682/) (2023)
- [Semantic-Aware Frame-Event Fusion based Pattern Recognition via Large Vision-Language Models](https://mlsanity.com/paper/2311.18592/) (2023)
- [E2VIDiff: Perceptual Events-to-Video Reconstruction using Diffusion Priors](https://mlsanity.com/paper/2407.08231/) (2024)
- [Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis](https://mlsanity.com/paper/2505.00627/) (2025)
- [EventMamba: Enhancing Spatio-Temporal Locality with State Space Models for Event-Based Video Reconstruction](https://mlsanity.com/paper/2503.19721/) (2025)
- [A Temporal Densely Connected Recurrent Network for Event-based Human Pose Estimation](https://mlsanity.com/paper/2209.07034/) (2022)

## Abstract (arXiv)

Event cameras provide a promising sensing modality for high-speed and high-dynamic-range vision by asynchronously capturing brightness changes. A fundamental task in event-based vision is event-to-video (E2V) reconstruction, which aims to recover intensity videos from event streams. Most existing E2V approaches formulate reconstruction as a temporal--spatial signal recovery problem, relying on temporal aggregation and spatial feature learning to infer intensity frames. While effective to some extent, this formulation overlooks a critical limitation of event data: due to the change-driven sensing mechanism, event streams are inherently semantically under-determined, lacking object-level structure and contextual information that are essential for faithful reconstruction. In this work, we revisit E2V from a semantic perspective and argue that effective reconstruction requires going beyond temporal and spatial modeling to explicitly account for missing semantic information. Based on this insight, we propose textitSemantic-E2VID, a semantic-enriched end-to-end E2V framework that reformulates reconstruction as a process of semantic learning, fusing and decoding. Our approach first performs semantic abstraction by bridging event representations with semantics extracted from a pretrained Segment Anything Model (SAM), while avoiding modality-induced feature drift. The learned semantics are then fused into the event latent space in a representation-compatible manner, enabling event features to capture object-level structure and contextual cues. Furthermore, semantic-aware supervision is introduced to explicitly guide the reconstruction process toward semantically meaningful regions, complementing conventional pixel-level and temporal objectives. Extensive experiments on six public benchmarks demonstrate that Semantic-E2VID consistently outperforms state-of-the-art E2V methods.