# Real-time event simulation with frame-based cameras

arXiv: https://arxiv.org/abs/2209.04634 · HTML: https://mlsanity.com/paper/2209.04634/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 10 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [EV-SegNet: Semantic Segmentation for Event-based Cameras](https://mlsanity.com/paper/1811.12039/) (2018)
- [Event-aided Direct Sparse Odometry](https://mlsanity.com/paper/2204.07640/) (2022)
- [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [Event-based Motion Segmentation with Spatio-Temporal Graph Cuts](https://mlsanity.com/paper/2012.08730/) (2020)
- [EventGAN: Leveraging Large Scale Image Datasets for Event Cameras](https://mlsanity.com/paper/1912.01584/) (2019)
- [Event-driven Vision and Control for UAVs on a Neuromorphic Chip](https://mlsanity.com/paper/2108.03694/) (2021)
- [Differentiable Event Stream Simulator for Non-Rigid 3D Tracking](https://mlsanity.com/paper/2104.15139/) (2021)
- [Towards Low-Latency High-Bandwidth Control of Quadrotors using Event Cameras](https://mlsanity.com/paper/1911.04553/) (2019)
- [Event Guided Depth Sensing](https://mlsanity.com/paper/2110.10505/) (2021)
- [Enhanced Frame and Event-Based Simulator and Event-Based Video Interpolation Network](https://mlsanity.com/paper/2112.09379/) (2021)

## Built on by (most cited first, 7 of 7)

- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Neuromorphic Event-based Facial Expression Recognition](https://mlsanity.com/paper/2304.06351/) (2023)
- [V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulation](https://mlsanity.com/paper/2505.16797/) (2025)
- [Event Camera Guided Visual Media Restoration & 3D Reconstruction: A Survey](https://mlsanity.com/paper/2509.09971/) (2025)
- [BiasBench: A reproducible benchmark for tuning the biases of event cameras](https://mlsanity.com/paper/2504.18235/) (2025)
- [FracEvent: Event-Camera Simulation via Fractional-Relaxation Pixel Dynamics](https://mlsanity.com/paper/2606.26636/) (2026)
- [EventSTU: Event-Guided Efficient Spatio-Temporal Understanding for Video Large Language Models](https://mlsanity.com/paper/2511.18920/) (2025)

## Abstract (arXiv)

Event cameras are becoming increasingly popular in robotics and computer vision due to their beneficial properties, e.g., high temporal resolution, high bandwidth, almost no motion blur, and low power consumption. However, these cameras remain expensive and scarce in the market, making them inaccessible to the majority. Using event simulators minimizes the need for real event cameras to develop novel algorithms. However, due to the computational complexity of the simulation, the event streams of existing simulators cannot be generated in real-time but rather have to be pre-calculated from existing video sequences or pre-rendered and then simulated from a virtual 3D scene. Although these offline generated event streams can be used as training data for learning tasks, all response time dependent applications cannot benefit from these simulators yet, as they still require an actual event camera. This work proposes simulation methods that improve the performance of event simulation by two orders of magnitude (making them real-time capable) while remaining competitive in the quality assessment.