# Event-based Monocular Dense Depth Estimation with Recurrent Transformers

arXiv: https://arxiv.org/abs/2212.02791 · HTML: https://mlsanity.com/paper/2212.02791/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation](https://mlsanity.com/paper/1804.01306/) (2018)
- [MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer](https://mlsanity.com/paper/2208.03543/) (2022)
- [A Survey on Deep Learning Techniques for Stereo-based Depth Estimation](https://mlsanity.com/paper/2006.02535/) (2020)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [Learning Monocular Dense Depth from Events](https://mlsanity.com/paper/2010.08350/) (2020)
- [Focus Is All You Need: Loss Functions For Event-based Vision](https://mlsanity.com/paper/1904.07235/) (2019)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark](https://mlsanity.com/paper/2108.03830/) (2021)
- [Towards Real-Time Monocular Depth Estimation for Robotics: A Survey](https://mlsanity.com/paper/2111.08600/) (2021)
- [Event Transformer. A sparse-aware solution for efficient event data processing](https://mlsanity.com/paper/2204.03355/) (2022)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation](https://mlsanity.com/paper/2109.01801/) (2021)
- [Depthformer : Multiscale Vision Transformer For Monocular Depth Estimation With Local Global Information Fusion](https://mlsanity.com/paper/2207.04535/) (2022)
- [Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers](https://mlsanity.com/paper/2112.09685/) (2021)
- [VDTR: Video Deblurring with Transformer](https://mlsanity.com/paper/2204.08023/) (2022)
- [HiMODE: A Hybrid Monocular Omnidirectional Depth Estimation Model](https://mlsanity.com/paper/2204.05007/) (2022)

## Built on by (most cited first, 14 of 14)

- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [State Space Models for Event Cameras](https://mlsanity.com/paper/2402.15584/) (2024)
- [GET: Group Event Transformer for Event-Based Vision](https://mlsanity.com/paper/2310.02642/) (2023)
- [EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting](https://mlsanity.com/paper/2405.14959/) (2024)
- [Event-based Simultaneous Localization and Mapping: A Comprehensive Survey](https://mlsanity.com/paper/2304.09793/) (2023)
- [SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events](https://mlsanity.com/paper/2309.12842/) (2023)
- [PCDepth: Pattern-based Complementary Learning for Monocular Depth Estimation by Best of Both Worlds](https://mlsanity.com/paper/2402.18925/) (2024)
- [A Novel Spike Transformer Network for Depth Estimation from Event Cameras via Cross-modality Knowledge Distillation](https://mlsanity.com/paper/2404.17335/) (2024)
- [Motion Segmentation for Neuromorphic Aerial Surveillance](https://mlsanity.com/paper/2405.15209/) (2024)
- [Dense Depth from Event Focal Stack](https://mlsanity.com/paper/2412.08120/) (2024)
- [DELTA: Dense Depth from Events and LiDAR using Transformer's Attention](https://mlsanity.com/paper/2505.02593/) (2025)
- [Event3R: Asynchronous-to-Global 3D Reconstruction from Event Camera via Spatial-Temporal Feature Aggregation](https://mlsanity.com/paper/2607.15727/) (2026)
- [Depth Hypothesis Guided Iterative Refinement for Event-Image Monocular Depth Estimation](https://mlsanity.com/paper/2610.03439/) (2026)

## Abstract (arXiv)

Event cameras, offering high temporal resolutions and high dynamic ranges, have brought a new perspective to address common challenges (e.g., motion blur and low light) in monocular depth estimation. However, how to effectively exploit the sparse spatial information and rich temporal cues from asynchronous events remains a challenging endeavor. To this end, we propose a novel event-based monocular depth estimator with recurrent transformers, namely EReFormer, which is the first pure transformer with a recursive mechanism to process continuous event streams. Technically, for spatial modeling, a novel transformer-based encoder-decoder with a spatial transformer fusion module is presented, having better global context information modeling capabilities than CNN-based methods. For temporal modeling, we design a gate recurrent vision transformer unit that introduces a recursive mechanism into transformers, improving temporal modeling capabilities while alleviating the expensive GPU memory cost. The experimental results show that our EReFormer outperforms state-of-the-art methods by a margin on both synthetic and real-world datasets. We hope that our work will attract further research to develop stunning transformers in the event-based vision community. Our open-source code can be found in the supplemental material.