# DELTA: Dense Depth from Events and LiDAR using Transformer's Attention

arXiv: https://arxiv.org/abs/2505.02593 · HTML: https://mlsanity.com/paper/2505.02593/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 52 references that are in the map.
- First posted on arXiv 5 May 2025.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [Learning Monocular Dense Depth from Events](https://mlsanity.com/paper/2010.08350/) (2020)
- [Time-Ordered Recent Event (TORE) Volumes for Event Cameras](https://mlsanity.com/paper/2103.06108/) (2021)
- [GET: Group Event Transformer for Event-Based Vision](https://mlsanity.com/paper/2310.02642/) (2023)
- [From Chaos Comes Order: Ordering Event Representations for Object Recognition and Detection](https://mlsanity.com/paper/2304.13455/) (2023)
- [Event Transformer. A sparse-aware solution for efficient event data processing](https://mlsanity.com/paper/2204.03355/) (2022)
- [Hierarchical Neural Memory Network for Low Latency Event Processing](https://mlsanity.com/paper/2305.17852/) (2023)
- [Exploiting Spatial Sparsity for Event Cameras with Visual Transformers](https://mlsanity.com/paper/2202.05054/) (2022)
- [StereoSpike: Depth Learning with a Spiking Neural Network](https://mlsanity.com/paper/2109.13751/) (2021)
- [Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion](https://mlsanity.com/paper/2207.10494/) (2022)
- [L2E: Lasers to Events for 6-DoF Extrinsic Calibration of Lidars and Event Cameras](https://mlsanity.com/paper/2207.01009/) (2022)
- [Event-based Monocular Dense Depth Estimation with Recurrent Transformers](https://mlsanity.com/paper/2212.02791/) (2022)
- [Event Transformer+. A multi-purpose solution for efficient event data processing](https://mlsanity.com/paper/2211.12222/) (2022)
- [LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With A Globally Optimal Solution](https://mlsanity.com/paper/2303.09825/) (2023)
- [PCDepth: Pattern-based Complementary Learning for Monocular Depth Estimation by Best of Both Worlds](https://mlsanity.com/paper/2402.18925/) (2024)

## Abstract (arXiv)

Event cameras and LiDARs provide complementary yet distinct data: respectively, asynchronous detections of changes in lighting versus sparse but accurate depth information at a fixed rate. To this day, few works have explored the combination of these two modalities. In this article, we propose a novel neural-network-based method for fusing event and LiDAR data in order to estimate dense depth maps. Our architecture, DELTA, exploits the concepts of self- and cross-attention to model the spatial and temporal relations within and between the event and LiDAR data. Following a thorough evaluation, we demonstrate that DELTA sets a new state of the art in the event-based depth estimation problem, and that it is able to reduce the errors up to four times for close ranges compared to the previous SOTA.