EVIS: Real-Time Event Camera Simulation with Multimodal Supervision in NVIDIA Isaac Sim
Not on the map. Why?
Abstract (arXiv)
Event cameras are increasingly adopted in embodied perception for their microsecond temporal resolution, high dynamic range, and resilience to motion blur. However, training event-based models for robotics requires large-scale, action-conditioned data with dense physical annotations that are difficult to collect in the real world. We introduce EVIS, an open-source physics-grounded event simulator integrated into NVIDIA Isaac Sim that generates events from linear-HDR radiance from a closed-loop robot training episode. Rather than relying on learning-based video interpolation or expensive dense rendering, EVIS exploits renderer-provided motion vectors and depth maps through bi-directional warping and depth-based splatting. This enables high throughput, real-time, and high-fidelity event generation. We evaluate EVIS across runtime efficiency, sim-to-real transfer, and zero-shot model compatibility. A rotation-speed estimator trained solely on EVIS events achieves 2.75 rad/s MAE on real sensor data. Pretrained models for reconstruction, matching, and tracking perform competitively on EVIS events without any fine-tuning. EVIS sustains real-time generation across GPU-parallel environments on a single GPU. Code repository: https://github.com/spikelab-jhu/isaac-sim-event-camera-plugin.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
31 of 46 references.
References · 31
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
- 2019Event-based Vision: A Surveycited by 1,010
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
- 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
- 2019High Speed and High Dynamic Range Video with an Event Cameracited by 344
- 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
- 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
- 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 211
19 more
- 2016The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAMcited by 203
- 2020Learning to Detect Objects with a 1 Megapixel Event Cameracited by 184
- 2019Video to Events: Recycling Video Datasets for Event Camerascited by 162
- 2022FILM: Frame Interpolation for Large Motioncited by 149
- 2019EventGAN: Leveraging Large Scale Image Datasets for Event Camerascited by 48
- 2020EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Streamcited by 48
- 2023Learning Optical Flow from Event Camera with Rendered Datasetcited by 26
- 2023V2CE: Video to Continuous Events Simulatorcited by 24
- 2023EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fieldscited by 17
- 2024Generalizable Implicit Motion Modeling for Video Frame Interpolationcited by 16
- 2025Depth AnyEvent: A Cross-Modal Distillation Paradigm for Event-Based Monocular Depth Estimationcited by 13
- 2024ETAP: Event-based Tracking of Any Pointcited by 9
- 2024ADV2E: Bridging the Gap Between Analogue Circuit and Discrete Frames in the Video-to-Events Simulatorcited by 7
- 2025How Real is CARLAs Dynamic Vision Sensor? A Study on the Sim-to-Real Gap in Traffic Object Detectioncited by 6
- 2025A PyTorch-Enabled Tool for Synthetic Event Camera Data Generation and Algorithm Developmentcited by 4
- 2025V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulationcited by 4
- 2025EventTracer: Fast Path Tracing-based Event Stream Renderingcited by 2
- 2024Monte Carlo Path Tracing and Statistical Event Detection for Event Camera Simulationcited by 1
- 2026Match-Any-Events: Zero-Shot Motion-Robust Feature Matching Across Wide Baselines for Event Camerascited by 1
Built on by
Not cited yet.