# EVIS: Real-Time Event Camera Simulation with Multimodal Supervision in NVIDIA Isaac Sim

arXiv: https://arxiv.org/abs/2607.08098 · HTML: https://mlsanity.com/paper/2607.08098/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 9 Jul 2026.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM](https://mlsanity.com/paper/1610.08336/) (2016)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)
- [EventGAN: Leveraging Large Scale Image Datasets for Event Cameras](https://mlsanity.com/paper/1912.01584/) (2019)
- [EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream](https://mlsanity.com/paper/2012.06475/) (2020)
- [Learning Optical Flow from Event Camera with Rendered Dataset](https://mlsanity.com/paper/2303.11011/) (2023)
- [V2CE: Video to Continuous Events Simulator](https://mlsanity.com/paper/2309.08891/) (2023)
- [EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields](https://mlsanity.com/paper/2310.02437/) (2023)
- [Generalizable Implicit Motion Modeling for Video Frame Interpolation](https://mlsanity.com/paper/2407.08680/) (2024)
- [Depth AnyEvent: A Cross-Modal Distillation Paradigm for Event-Based Monocular Depth Estimation](https://mlsanity.com/paper/2509.15224/) (2025)
- [ETAP: Event-based Tracking of Any Point](https://mlsanity.com/paper/2412.00133/) (2024)
- [ADV2E: Bridging the Gap Between Analogue Circuit and Discrete Frames in the Video-to-Events Simulator](https://mlsanity.com/paper/2411.12250/) (2024)
- [How Real is CARLAs Dynamic Vision Sensor? A Study on the Sim-to-Real Gap in Traffic Object Detection](https://mlsanity.com/paper/2506.13722/) (2025)
- [A PyTorch-Enabled Tool for Synthetic Event Camera Data Generation and Algorithm Development](https://mlsanity.com/paper/2503.09754/) (2025)
- [V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulation](https://mlsanity.com/paper/2505.16797/) (2025)
- [EventTracer: Fast Path Tracing-based Event Stream Rendering](https://mlsanity.com/paper/2508.18071/) (2025)
- [Monte Carlo Path Tracing and Statistical Event Detection for Event Camera Simulation](https://mlsanity.com/paper/2408.07996/) (2024)

## Abstract (arXiv)

Event cameras are increasingly adopted in embodied perception for their microsecond temporal resolution, high dynamic range, and resilience to motion blur. However, training event-based models for robotics requires large-scale, action-conditioned data with dense physical annotations that are difficult to collect in the real world. We introduce EVIS, an open-source physics-grounded event simulator integrated into NVIDIA Isaac Sim that generates events from linear-HDR radiance from a closed-loop robot training episode. Rather than relying on learning-based video interpolation or expensive dense rendering, EVIS exploits renderer-provided motion vectors and depth maps through bi-directional warping and depth-based splatting. This enables high throughput, real-time, and high-fidelity event generation. We evaluate EVIS across runtime efficiency, sim-to-real transfer, and zero-shot model compatibility. A rotation-speed estimator trained solely on EVIS events achieves 2.75 rad/s MAE on real sensor data. Pretrained models for reconstruction, matching, and tracking perform competitively on EVIS events without any fine-tuning. EVIS sustains real-time generation across GPU-parallel environments on a single GPU. Code repository: https://github.com/spikelab-jhu/isaac-sim-event-camera-plugin.