# Emergent Visual Sensors for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2205.09383 · HTML: https://mlsanity.com/paper/2205.09383/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 247 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 30 of 70)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [End-to-end Lane Shape Prediction with Transformers](https://mlsanity.com/paper/2011.04233/) (2020)
- [A Large Scale Event-based Detection Dataset for Automotive](https://mlsanity.com/paper/2001.08499/) (2020)
- [Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems](https://mlsanity.com/paper/2004.08467/) (2020)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [EV-SegNet: Semantic Segmentation for Event-based Cameras](https://mlsanity.com/paper/1811.12039/) (2018)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [Cross-Modality Fusion Transformer for Multispectral Object Detection](https://mlsanity.com/paper/2111.00273/) (2021)

## Built on by (most cited first, 19 of 19)

- [Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion](https://mlsanity.com/paper/2311.00436/) (2023)
- [Distilling Diffusion Models to Efficient 3D LiDAR Scene Completion](https://mlsanity.com/paper/2412.03515/) (2024)
- [MULi-Ev: Maintaining Unperturbed LiDAR-Event Calibration](https://mlsanity.com/paper/2405.18021/) (2024)
- [Talk to Parallel LiDARs: A Human-LiDAR Interaction Method Based on 3D Visual Grounding](https://mlsanity.com/paper/2405.15274/) (2024)
- [Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-View](https://mlsanity.com/paper/2402.00637/) (2024)
- [Optical Flow Matters: an Empirical Comparative Study on Fusing Monocular Extracted Modalities for Better Steering](https://mlsanity.com/paper/2409.12716/) (2024)
- [Multi-Timescale Motion-Decoupled Spiking Transformer for Audio-Visual Zero-Shot Learning](https://mlsanity.com/paper/2505.19938/) (2025)
- [EANet: Enhanced Attribute-based RGBT Tracker Network](https://mlsanity.com/paper/2307.01893/) (2023)
- [Camera-Based Localization and Enhanced Normalized Mutual Information](https://mlsanity.com/paper/2412.16137/) (2024)
- [Planar Velocity Estimation for Fast-Moving Mobile Robots Using Event-Based Optical Flow](https://mlsanity.com/paper/2505.11116/) (2025)
- [EIFNet: Leveraging Event-Image Fusion for Robust Semantic Segmentation](https://mlsanity.com/paper/2507.21971/) (2025)
- [A Survey on Deep Multi-Task Learning in Connected Autonomous Vehicles](https://mlsanity.com/paper/2508.00917/) (2025)
- [MSGFusion: Multimodal Scene Graph-Guided Infrared and Visible Image Fusion](https://mlsanity.com/paper/2509.12901/) (2025)
- [No Calibration, No Depth, No Problem: Cross-Sensor View Synthesis with 3D Consistency](https://mlsanity.com/paper/2602.23559/) (2026)
- [PatchScene: Patch-based Voxel Diffusion for Large-Scale Scene Completion](https://mlsanity.com/paper/2606.03915/) (2026)
- [DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data](https://mlsanity.com/paper/2606.27576/) (2026)
- [Image Quality Dependent Degradation for AI Systems](https://mlsanity.com/paper/2607.25736/) (2026)
- [CORAL: Curriculum-Optimized Reward Adaptation for LiDAR-Based Goal-Directed Urban Driving](https://mlsanity.com/paper/2608.14332/) (2026)
- [SenseWeather: Multimodal Sensing for Robust Robotic Perception under Extreme Conditions](https://mlsanity.com/paper/2610.14082/) (2026)

## Abstract (arXiv)

Autonomous vehicles rely on perception systems to understand their surroundings for further navigation missions. Cameras are essential for perception systems due to the advantages of object detection and recognition provided by modern computer vision algorithms, comparing to other sensors, such as LiDARs and radars. However, limited by its inherent imaging principle, a standard RGB camera may perform poorly in a variety of adverse scenarios, including but not limited to: low illumination, high contrast, bad weather such as fog/rain/snow, etc. Meanwhile, estimating the 3D information from the 2D image detection is generally more difficult when compared to LiDARs or radars. Several new sensing technologies have emerged in recent years to address the limitations of conventional RGB cameras. In this paper, we review the principles of four novel image sensors: infrared cameras, range-gated cameras, polarization cameras, and event cameras. Their comparative advantages, existing or potential applications, and corresponding data processing algorithms are all presented in a systematic manner. We expect that this study will assist practitioners in the autonomous driving society with new perspectives and insights.