# Stereo Hybrid Event-Frame (SHEF) Cameras for 3D Perception

arXiv: https://arxiv.org/abs/2110.04988 · HTML: https://mlsanity.com/paper/2110.04988/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [Continuous-time Intensity Estimation Using Event Cameras](https://mlsanity.com/paper/1811.00386/) (2018)
- [Semi-Dense 3D Reconstruction with a Stereo Event Camera](https://mlsanity.com/paper/1807.07429/) (2018)
- [EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras](https://mlsanity.com/paper/1903.07520/) (2019)
- [Single Image Optical Flow Estimation with an Event Camera](https://mlsanity.com/paper/2004.00347/) (2020)
- [Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restoration](https://mlsanity.com/paper/2012.00301/) (2020)
- [AutoDispNet: Improving Disparity Estimation With AutoML](https://mlsanity.com/paper/1905.07443/) (2019)
- [An Asynchronous Kalman Filter for Hybrid Event Cameras](https://mlsanity.com/paper/2012.05590/) (2020)
- [Event Camera Calibration of Per-pixel Biased Contrast Threshold](https://mlsanity.com/paper/2012.09378/) (2020)

## Built on by (most cited first, 17 of 17)

- [Ev-NeRF: Event Based Neural Radiance Field](https://mlsanity.com/paper/2206.12455/) (2022)
- [Event Transformer. A sparse-aware solution for efficient event data processing](https://mlsanity.com/paper/2204.03355/) (2022)
- [Event-based Stereo Depth Estimation: A Survey](https://mlsanity.com/paper/2409.17680/) (2024)
- [Asynchronous Blob Tracker for Event Cameras](https://mlsanity.com/paper/2307.10593/) (2023)
- [Event Transformer+. A multi-purpose solution for efficient event data processing](https://mlsanity.com/paper/2211.12222/) (2022)
- [Data-Driven Feature Tracking for Event Cameras With and Without Frames](https://mlsanity.com/paper/2211.12826/) (2022)
- [Video Frame Interpolation with Stereo Event and Intensity Camera](https://mlsanity.com/paper/2307.08228/) (2023)
- [EventRPG: Event Data Augmentation with Relevance Propagation Guidance](https://mlsanity.com/paper/2403.09274/) (2024)
- [An Asynchronous Linear Filter Architecture for Hybrid Event-Frame Cameras](https://mlsanity.com/paper/2309.01159/) (2023)
- [HUE Dataset: High-Resolution Event and Frame Sequences for Low-Light Vision](https://mlsanity.com/paper/2410.19164/) (2024)
- [Learning Parallax for Stereo Event-based Motion Deblurring](https://mlsanity.com/paper/2309.09513/) (2023)
- [Tracking Fast by Learning Slow: An Event-based Speed Adaptive Hand Tracker Leveraging Knowledge in RGB Domain](https://mlsanity.com/paper/2302.14430/) (2023)
- [Event-based Sensor Fusion and Application on Odometry: A Survey](https://mlsanity.com/paper/2410.15480/) (2024)
- [Match-Any-Events: Zero-Shot Motion-Robust Feature Matching Across Wide Baselines for Event Cameras](https://mlsanity.com/paper/2604.18744/) (2026)
- [EventHub: Data Factory for Generalizable Event-Based Stereo Networks without Active Sensors](https://mlsanity.com/paper/2604.02331/) (2026)
- [Bidirectional Cross-Modal Prompting for Event-Frame Asymmetric Stereo](https://mlsanity.com/paper/2604.15312/) (2026)
- [Asynchronous Tracking, Optical Communication and 3D Motion Capture using Event-based Sensors](https://mlsanity.com/paper/2610.04342/) (2026)

## Abstract (arXiv)

Stereo camera systems play an important role in robotics applications to perceive the 3D world. However, conventional cameras have drawbacks such as low dynamic range, motion blur and latency due to the underlying frame-based mechanism. Event cameras address these limitations as they report the brightness changes of each pixel independently with a fine temporal resolution, but they are unable to acquire absolute intensity information directly. Although integrated hybrid event-frame sensors (eg., DAVIS) are available, the quality of data is compromised by coupling at the pixel level in the circuit fabrication of such cameras. This paper proposes a stereo hybrid event-frame (SHEF) camera system that offers a sensor modality with separate high-quality pure event and pure frame cameras, overcoming the limitations of each separate sensor and allowing for stereo depth estimation. We provide a SHEF dataset targeted at evaluating disparity estimation algorithms and introduce a stereo disparity estimation algorithm that uses edge information extracted from the event stream correlated with the edge detected in the frame data. Our disparity estimation outperforms the state-of-the-art stereo matching algorithm on the SHEF dataset.