# An Asynchronous Linear Filter Architecture for Hybrid Event-Frame Cameras

arXiv: https://arxiv.org/abs/2309.01159 · HTML: https://mlsanity.com/paper/2309.01159/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 94 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion](https://mlsanity.com/paper/2203.17191/) (2022)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [Continuous-time Intensity Estimation Using Event Cameras](https://mlsanity.com/paper/1811.00386/) (2018)
- [EventNet: Asynchronous Recursive Event Processing](https://mlsanity.com/paper/1812.07045/) (2018)
- [Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisation](https://mlsanity.com/paper/1607.06283/) (2016)
- [CED: Color Event Camera Dataset](https://mlsanity.com/paper/1904.10772/) (2019)
- [Multi-Bracket High Dynamic Range Imaging with Event Cameras](https://mlsanity.com/paper/2203.06622/) (2022)
- [Moving Object Detection for Event-based vision using Graph Spectral Clustering](https://mlsanity.com/paper/2109.14979/) (2021)
- [An Asynchronous Kalman Filter for Hybrid Event Cameras](https://mlsanity.com/paper/2012.05590/) (2020)
- [Event Camera Calibration of Per-pixel Biased Contrast Threshold](https://mlsanity.com/paper/2012.09378/) (2020)
- [Asynchronous Spatial Image Convolutions for Event Cameras](https://mlsanity.com/paper/1812.00438/) (2018)
- [Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception](https://mlsanity.com/paper/2110.04988/) (2021)
- [Asynchronous Blob Tracker for Event Cameras](https://mlsanity.com/paper/2307.10593/) (2023)
- [A Linear Comb Filter for Event Flicker Removal](https://mlsanity.com/paper/2205.08090/) (2022)
- [Event-driven Video Frame Synthesis](https://mlsanity.com/paper/1902.09680/) (2019)
- [Smart Visual Beacons with Asynchronous Optical Communications using Event Cameras](https://mlsanity.com/paper/2208.01710/) (2022)
- [Learn to See by Events: Color Frame Synthesis from Event and RGB Cameras](https://mlsanity.com/paper/1812.02041/) (2018)
- [Photorealistic Image Reconstruction from Hybrid Intensity and Event based Sensor](https://mlsanity.com/paper/1805.06140/) (2018)

## Built on by (most cited first, 6 of 6)

- [Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging](https://mlsanity.com/paper/2404.11511/) (2024)
- [Asynchronous Multi-Object Tracking with an Event Camera](https://mlsanity.com/paper/2505.08126/) (2025)
- [An Event-Oriented Diffusion-Refinement Method for Sparse Events Completion](https://mlsanity.com/paper/2401.03153/) (2024)
- [A Label-Free and Non-Monotonic Metric for Evaluating Denoising in Event Cameras](https://mlsanity.com/paper/2406.08909/) (2024)
- [A dynamic vision sensor object recognition model based on trainable event-driven convolution and spiking attention mechanism](https://mlsanity.com/paper/2409.12691/) (2024)
- [Seeing through Light and Darkness: Sensor-Physics Grounded Deblurring HDR NeRF from Single-Exposure Images and Events](https://mlsanity.com/paper/2601.15475/) (2026)

## Abstract (arXiv)

Event cameras are ideally suited to capture High Dynamic Range (HDR) visual information without blur but provide poor imaging capability for static or slowly varying scenes. Conversely, conventional image sensors measure absolute intensity of slowly changing scenes effectively but do poorly on HDR or quickly changing scenes. In this paper, we present an asynchronous linear filter architecture, fusing event and frame camera data, for HDR video reconstruction and spatial convolution that exploits the advantages of both sensor modalities. The key idea is the introduction of a state that directly encodes the integrated or convolved image information and that is updated asynchronously as each event or each frame arrives from the camera. The state can be read-off as-often-as and whenever required to feed into subsequent vision modules for real-time robotic systems. Our experimental results are evaluated on both publicly available datasets with challenging lighting conditions and fast motions, along with a new dataset with HDR reference that we provide. The proposed AKF pipeline outperforms other state-of-the-art methods in both absolute intensity error (69.4% reduction) and image similarity indexes (average 35.5% improvement). We also demonstrate the integration of image convolution with linear spatial kernels Gaussian, Sobel, and Laplacian as an application of our architecture.