# V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulation

arXiv: https://arxiv.org/abs/2505.16797 · HTML: https://mlsanity.com/paper/2505.16797/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks](https://mlsanity.com/paper/2003.06696/) (2020)
- [Learning Dense and Continuous Optical Flow from an Event Camera](https://mlsanity.com/paper/2211.09078/) (2022)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation](https://mlsanity.com/paper/2109.04871/) (2021)
- [EventGAN: Leveraging Large Scale Image Datasets for Event Cameras](https://mlsanity.com/paper/1912.01584/) (2019)
- [HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks](https://mlsanity.com/paper/2305.06382/) (2023)
- [Seeing Motion at Nighttime with an Event Camera](https://mlsanity.com/paper/2404.11884/) (2024)
- [BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation](https://mlsanity.com/paper/2303.07716/) (2023)
- [Learning Optical Flow from Event Camera with Rendered Dataset](https://mlsanity.com/paper/2303.11011/) (2023)
- [V2CE: Video to Continuous Events Simulator](https://mlsanity.com/paper/2309.08891/) (2023)
- [Perception-Oriented Video Frame Interpolation via Asymmetric Blending](https://mlsanity.com/paper/2404.06692/) (2024)
- [EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset](https://mlsanity.com/paper/2312.08220/) (2023)
- [Real-time event simulation with frame-based cameras](https://mlsanity.com/paper/2209.04634/) (2022)

## Built on by (most cited first, 4 of 4)

- [EventSTU: Event-Guided Efficient Spatio-Temporal Understanding for Video Large Language Models](https://mlsanity.com/paper/2511.18920/) (2025)
- [Reconstruction as a Bridge for Event-Based Visual Question Answering](https://mlsanity.com/paper/2512.11510/) (2025)
- [EVIS: Real-Time Event Camera Simulation with Multimodal Supervision in NVIDIA Isaac Sim](https://mlsanity.com/paper/2607.08098/) (2026)
- [Event ActivityNet: A Large-Scale Simulated-Event Benchmark for Untrimmed Action Understanding](https://mlsanity.com/paper/2608.01948/) (2026)

## Abstract (arXiv)

Event-based cameras offer unique advantages such as high temporal resolution, high dynamic range, and low power consumption. However, the massive storage requirements and I/O burdens of existing synthetic data generation pipelines and the scarcity of real data prevent event-based training datasets from scaling up, limiting the development and generalization capabilities of event vision models. To address this challenge, we introduce Video-to-Voxel (V2V), an approach that directly converts conventional video frames into event-based voxel grid representations, bypassing the storage-intensive event stream generation entirely. V2V enables a 150 times reduction in storage requirements while supporting on-the-fly parameter randomization for enhanced model robustness. Leveraging this efficiency, we train several video reconstruction and optical flow estimation model architectures on 10,000 diverse videos totaling 52 hours--an order of magnitude larger than existing event datasets, yielding substantial improvements.