# Event-driven Video Frame Synthesis

arXiv: https://arxiv.org/abs/1902.09680 · HTML: https://mlsanity.com/paper/1902.09680/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2019.

## Built on by (most cited first, 8 of 8)

- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [Splatting-based Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/2201.10075/) (2022)
- [Learning to Super-Resolve Blurry Images with Events](https://mlsanity.com/paper/2302.13766/) (2023)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [Noise2Image: Noise-Enabled Static Scene Recovery for Event Cameras](https://mlsanity.com/paper/2404.01298/) (2024)
- [An Asynchronous Linear Filter Architecture for Hybrid Event-Frame Cameras](https://mlsanity.com/paper/2309.01159/) (2023)
- [DeblurSR: Event-Based Motion Deblurring Under the Spiking Representation](https://mlsanity.com/paper/2303.08977/) (2023)
- [Removing Blocking Artifacts in Video Streams Using Event Cameras](https://mlsanity.com/paper/2105.05973/) (2021)

## Abstract (arXiv)

Temporal Video Frame Synthesis (TVFS) aims at synthesizing novel frames at timestamps different from existing frames, which has wide applications in video codec, editing and analysis. In this paper, we propose a high framerate TVFS framework which takes hybrid input data from a low-speed frame-based sensor and a high-speed event-based sensor. Compared to frame-based sensors, event-based sensors report brightness changes at very high speed, which may well provide useful spatio-temoral information for high framerate TVFS. In our framework, we first introduce a differentiable forward model to approximate the physical sensing process, fusing the two different modes of data as well as unifying a variety of TVFS tasks, i.e., interpolation, prediction and motion deblur. We leverage autodifferentiation which propagates the gradients of a loss defined on the measured data back to the latent high framerate video. We show results with better performance compared to state-of-the-art. Second, we develop a deep learning-based strategy to enhance the results from the first step, which we refer as a residual "denoising" process. Our trained "denoiser" is beyond Gaussian denoising and shows properties such as contrast enhancement and motion awareness. We show that our framework is capable of handling challenging scenes including both fast motion and strong occlusions.