# Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution

arXiv: https://arxiv.org/abs/2303.13767 · HTML: https://mlsanity.com/paper/2303.13767/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 50 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2108.11084/) (2021)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks](https://mlsanity.com/paper/1710.01992/) (2017)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Video Super-resolution with Temporal Group Attention](https://mlsanity.com/paper/2007.10595/) (2020)
- [Event Enhanced High-Quality Image Recovery](https://mlsanity.com/paper/2007.08336/) (2020)
- [Video Super Resolution Based on Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2007.12928/) (2020)
- [EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation](https://mlsanity.com/paper/2111.12341/) (2021)
- [Deep Video Super-Resolution using HR Optical Flow Estimation](https://mlsanity.com/paper/2001.02129/) (2020)
- [EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning](https://mlsanity.com/paper/2003.07640/) (2020)
- [CED: Color Event Camera Dataset](https://mlsanity.com/paper/1904.10772/) (2019)
- [Fast Spatio-Temporal Residual Network for Video Super-Resolution](https://mlsanity.com/paper/1904.02870/) (2019)

## Built on by (most cited first, 30 of 33)

- [Make Pixels Dance: High-Dynamic Video Generation](https://mlsanity.com/paper/2311.10982/) (2023)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [INCODE: Implicit Neural Conditioning with Prior Knowledge Embeddings](https://mlsanity.com/paper/2310.18846/) (2023)
- [Self-Supervised Facial Representation Learning with Facial Region Awareness](https://mlsanity.com/paper/2403.02138/) (2024)
- [Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction](https://mlsanity.com/paper/2403.07346/) (2024)
- [SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events](https://mlsanity.com/paper/2309.12842/) (2023)
- [UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation](https://mlsanity.com/paper/2305.15078/) (2023)
- [SNeRV: Spectra-preserving Neural Representation for Video](https://mlsanity.com/paper/2501.01681/) (2025)
- [EvTexture: Event-driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2406.13457/) (2024)
- [Superpixel-informed Implicit Neural Representation for Multi-Dimensional Data](https://mlsanity.com/paper/2411.11356/) (2024)
- [RGB-Event based Pedestrian Attribute Recognition: A Benchmark Dataset and An Asymmetric RWKV Fusion Framework](https://mlsanity.com/paper/2504.10018/) (2025)
- [HR-INR: Continuous Space-Time Video Super-Resolution via Event Camera](https://mlsanity.com/paper/2405.13389/) (2024)
- [Self-supervised Learning of Event-guided Video Frame Interpolation for Rolling Shutter Frames](https://mlsanity.com/paper/2306.15507/) (2023)
- [H-SIREN: Improving implicit neural representations with hyperbolic periodic functions](https://mlsanity.com/paper/2410.04716/) (2024)
- [ERetinex: Event Camera Meets Retinex Theory for Low-Light Image Enhancement](https://mlsanity.com/paper/2503.02484/) (2025)
- [Event Camera Demosaicing via Swin Transformer and Pixel-focus Loss](https://mlsanity.com/paper/2404.02731/) (2024)
- [Bilateral Event Mining and Complementary for Event Stream Super-Resolution](https://mlsanity.com/paper/2405.10037/) (2024)
- [Event Camera Guided Visual Media Restoration & 3D Reconstruction: A Survey](https://mlsanity.com/paper/2509.09971/) (2025)
- [ST-NeRP: Spatial-Temporal Neural Representation Learning with Prior Embedding for Patient-specific Imaging Study](https://mlsanity.com/paper/2410.19283/) (2024)
- [Event USKT : U-State Space Model in Knowledge Transfer for Event Cameras](https://mlsanity.com/paper/2411.15276/) (2024)
- [EvRWKV: A Continuous Interactive RWKV Framework for Effective Event-Guided Low-Light Image Enhancement](https://mlsanity.com/paper/2507.03184/) (2025)
- [Event-Stream Super Resolution using Sigma-Delta Neural Network](https://mlsanity.com/paper/2408.06968/) (2024)
- [Treat Stillness with Movement: Remote Sensing Change Detection via Coarse-grained Temporal Foregrounds Mining](https://mlsanity.com/paper/2408.08078/) (2024)
- [FANeRV: Frequency Separation and Augmentation based Neural Representation for Video](https://mlsanity.com/paper/2504.06755/) (2025)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [From Events to Enhancement: A Survey on Event-Based Imaging Technologies](https://mlsanity.com/paper/2505.05488/) (2025)
- [Learning Event-guided Exposure-agnostic Video Frame Interpolation via Adaptive Feature Blending](https://mlsanity.com/paper/2510.22565/) (2025)
- [From Events to Clarity: The Event-Guided Diffusion Framework for Dehazing](https://mlsanity.com/paper/2511.11944/) (2025)
- [Uncertainty-Calibrated Spatiotemporal Field Diffusion with Sparse Supervision](https://mlsanity.com/paper/2603.04431/) (2026)
- [High-Quality and Efficient Turbulence Mitigation with Events](https://mlsanity.com/paper/2603.20708/) (2026)

## Abstract (arXiv)

Event cameras sense the intensity changes asynchronously and produce event streams with high dynamic range and low latency. This has inspired research endeavors utilizing events to guide the challenging video superresolution (VSR) task. In this paper, we make the first attempt to address a novel problem of achieving VSR at random scales by taking advantages of the high temporal resolution property of events. This is hampered by the difficulties of representing the spatial-temporal information of events when guiding VSR. To this end, we propose a novel framework that incorporates the spatial-temporal interpolation of events to VSR in a unified framework. Our key idea is to learn implicit neural representations from queried spatial-temporal coordinates and features from both RGB frames and events. Our method contains three parts. Specifically, the Spatial-Temporal Fusion (STF) module first learns the 3D features from events and RGB frames. Then, the Temporal Filter (TF) module unlocks more explicit motion information from the events near the queried timestamp and generates the 2D features. Lastly, the SpatialTemporal Implicit Representation (STIR) module recovers the SR frame in arbitrary resolutions from the outputs of these two modules. In addition, we collect a real-world dataset with spatially aligned events and RGB frames. Extensive experiments show that our method significantly surpasses the prior-arts and achieves VSR with random scales, e.g., 6.5. Code and dataset are available at https: //vlis2022.github.io/cvpr23/egvsr.