# How Asynchronous Events Encode Video

arXiv: https://arxiv.org/abs/2206.04341 · HTML: https://mlsanity.com/paper/2206.04341/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 8 of 8)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Need for Speed: A Benchmark for Higher Frame Rate Object Tracking](https://mlsanity.com/paper/1703.05884/) (2017)
- [Reconstructing Classes of Non-bandlimited Signals from Time Encoded Information](https://mlsanity.com/paper/1905.03183/) (2019)
- [Asynchrony Increases Efficiency: Time Encoding of Videos and Low-Rank Signals](https://mlsanity.com/paper/2104.14511/) (2021)
- [Encoding and Decoding Mixed Bandlimited Signals using Spiking Integrate-and-Fire Neurons](https://mlsanity.com/paper/1910.09413/) (2019)
- [Matrix recovery from bilinear and quadratic measurements](https://mlsanity.com/paper/2001.04933/) (2020)
- [A Time Encoding approach to training Spiking Neural Networks](https://mlsanity.com/paper/2110.06735/) (2021)

## Abstract (arXiv)

As event-based sensing gains in popularity, theoretical understanding is needed to harness this technology's potential. Instead of recording video by capturing frames, event-based cameras have sensors that emit events when their inputs change, thus encoding information in the timing of events. This creates new challenges in establishing reconstruction guarantees and algorithms, but also provides advantages over frame-based video. We use time encoding machines to model event-based sensors: TEMs also encode their inputs by emitting events characterized by their timing and reconstruction from time encodings is well understood. We consider the case of time encoding bandlimited video and demonstrate a dependence between spatial sensor density and overall spatial and temporal resolution. Such a dependence does not occur in frame-based video, where temporal resolution depends solely on the frame rate of the video and spatial resolution depends solely on the pixel grid. However, this dependence arises naturally in event-based video and allows oversampling in space to provide better time resolution. As such, event-based vision encourages using more sensors that emit fewer events over time.