# A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild

arXiv: https://arxiv.org/abs/2301.05191 · HTML: https://mlsanity.com/paper/2301.05191/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 70 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model](https://mlsanity.com/paper/1706.03458/) (2017)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Event-Based Fusion for Motion Deblurring with Cross-modal Attention](https://mlsanity.com/paper/2112.00167/) (2021)
- [Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion](https://mlsanity.com/paper/2203.17191/) (2022)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1811.08230/) (2018)
- [Secrets of Event-Based Optical Flow](https://mlsanity.com/paper/2207.10022/) (2022)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [Unifying Motion Deblurring and Frame Interpolation with Events](https://mlsanity.com/paper/2203.12178/) (2022)

## Built on by (most cited first, 27 of 27)

- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Mitigating Motion Blur in Neural Radiance Fields with Events and Frames](https://mlsanity.com/paper/2403.19780/) (2024)
- [Towards Real-world Event-guided Low-light Video Enhancement and Deblurring](https://mlsanity.com/paper/2408.14916/) (2024)
- [EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset](https://mlsanity.com/paper/2312.08220/) (2023)
- [NTIRE 2025 Challenge on Event-Based Image Deblurring: Methods and Results](https://mlsanity.com/paper/2504.12401/) (2025)
- [BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream](https://mlsanity.com/paper/2407.02174/) (2024)
- [Multi-task Image Restoration Guided By Robust DINO Features](https://mlsanity.com/paper/2312.01677/) (2023)
- [Temporal Event Stereo via Joint Learning with Stereoscopic Flow](https://mlsanity.com/paper/2407.10831/) (2024)
- [Low-Light Image Enhancement using Event-Based Illumination Estimation](https://mlsanity.com/paper/2504.09379/) (2025)
- [Video Frame Interpolation with Stereo Event and Intensity Camera](https://mlsanity.com/paper/2307.08228/) (2023)
- [Temporal-Mapping Photography for Event Cameras](https://mlsanity.com/paper/2403.06443/) (2024)
- [CMTA: Cross-Modal Temporal Alignment for Event-guided Video Deblurring](https://mlsanity.com/paper/2408.14930/) (2024)
- [From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization](https://mlsanity.com/paper/2406.08090/) (2024)
- [Event-Based Motion Magnification](https://mlsanity.com/paper/2402.11957/) (2024)
- [SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams](https://mlsanity.com/paper/2403.09486/) (2024)
- [Improving Fast Auto-Focus with Event Polarity](https://mlsanity.com/paper/2303.08611/) (2023)
- [Segment Any Events via Weighted Adaptation of Pivotal Tokens](https://mlsanity.com/paper/2312.16222/) (2023)
- [TimeRewind: Rewinding Time with Image-and-Events Video Diffusion](https://mlsanity.com/paper/2403.13800/) (2024)
- [Learning to Remove Lens Flare in Event Camera](https://mlsanity.com/paper/2512.09016/) (2025)
- [An Event-Oriented Diffusion-Refinement Method for Sparse Events Completion](https://mlsanity.com/paper/2401.03153/) (2024)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [EVDI++: Event-based Video Deblurring and Interpolation via Self-Supervised Learning](https://mlsanity.com/paper/2509.08260/) (2025)
- [EvDiff: Event-Based Video Reconstruction using One-Step Diffusion Models](https://mlsanity.com/paper/2511.17492/) (2025)
- [Reconstruction as a Bridge for Event-Based Visual Question Answering](https://mlsanity.com/paper/2512.11510/) (2025)
- [EvTexture++: Event-Driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2606.13580/) (2026)
- [LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models](https://mlsanity.com/paper/2607.08770/) (2026)
- [Event-Based Motion Estimation via Oriented Distance Fields](https://mlsanity.com/paper/2608.24223/) (2026)

## Abstract (arXiv)

Effective video frame interpolation hinges on the adept handling of motion in the input scene. Prior work acknowledges asynchronous event information for this, but often overlooks whether motion induces blur in the video, limiting its scope to sharp frame interpolation. We instead propose a unified framework for event-based frame interpolation that performs deblurring ad-hoc and thus works both on sharp and blurry input videos. Our model consists in a bidirectional recurrent network that incorporates the temporal dimension of interpolation and fuses information from the input frames and the events adaptively based on their temporal proximity. To enhance the generalization from synthetic data to real event cameras, we integrate self-supervised framework with the proposed model to enhance the generalization on real-world datasets in the wild. At the dataset level, we introduce a novel real-world high-resolution dataset with events and color videos named HighREV, which provides a challenging evaluation setting for the examined task. Extensive experiments show that our network consistently outperforms previous state-of-the-art methods on frame interpolation, single image deblurring, and the joint task of both. Experiments on domain transfer reveal that self-supervised training effectively mitigates the performance degradation observed when transitioning from synthetic data to real-world data. Code and datasets are available at https://github.com/AHupuJR/REFID.