# Engram-E2VID: Reference-Based Event-to-Video Reconstruction via Generative Activation of Appearance Engrams

arXiv: https://arxiv.org/abs/2608.05728 · HTML: https://mlsanity.com/paper/2608.05728/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 20 of 20)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion](https://mlsanity.com/paper/2203.17191/) (2022)
- [Learning Dense and Continuous Optical Flow from an Event Camera](https://mlsanity.com/paper/2211.09078/) (2022)
- [A Dynamic Multi-Scale Voxel Flow Network for Video Prediction](https://mlsanity.com/paper/2303.09875/) (2023)
- [HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks](https://mlsanity.com/paper/2305.06382/) (2023)
- [Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fields](https://mlsanity.com/paper/2502.13716/) (2025)
- [Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolation](https://mlsanity.com/paper/2412.07761/) (2024)
- [EventHDR: from Event to High-Speed HDR Videos and Beyond](https://mlsanity.com/paper/2409.17029/) (2024)
- [Event-based Continuous Color Video Decompression from Single Frames](https://mlsanity.com/paper/2312.00113/) (2023)
- [Temporal Residual Guided Diffusion Framework for Event-Driven Video Reconstruction](https://mlsanity.com/paper/2407.10636/) (2024)
- [Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flow](https://mlsanity.com/paper/2503.06992/) (2025)
- [Motion Graph Unleashed: A Novel Approach to Video Prediction](https://mlsanity.com/paper/2410.22288/) (2024)
- [DESSERT: Diffusion-based Event-driven Single-frame Synthesis via Residual Training](https://mlsanity.com/paper/2512.17323/) (2025)

## Abstract (arXiv)

Reference-based event-to-video reconstruction aims to recover target RGB frames from a reference frame and the event stream captured over the reference-to-target interval. Although events provide fine-grained temporal cues, they encode sparse and asynchronous log-intensity changes rather than absolute appearance, making faithful reconstruction intrinsically challenging. The central challenge lies in associating event-derived target-time structures with relevant appearance information from the reference frame, especially under complex motion and long temporal intervals. In this work, we propose Engram-E2VID, a structure-guided framework that reconstructs target frames through the generative activation of appearance engrams. Specifically, the reference frame is encoded into token-space appearance engrams, while the event stream and reference context are transformed into a target-time motion-structure scaffold that captures motion boundaries and event-induced structural changes. Within a one-step diffusion backbone, scaffold-derived structural tokens progressively interact with and activate relevant appearance engrams across layers. This token-space association allows target structures to access reference appearance without relying on direct pixel-wise correspondence, while the diffusion prior complements uncertain or newly revealed regions. Across three benchmarks, Engram-E2VID improves PSNR by up to 3.29 dB and reduces LPIPS by up to 0.08 over the strongest same-input baseline, while degrading more slowly as the reconstruction interval increases.