# Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data

arXiv: https://arxiv.org/abs/1809.08625 · HTML: https://mlsanity.com/paper/1809.08625/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [Globally-Optimal Contrast Maximisation for Event Cameras](https://mlsanity.com/paper/2206.05127/) (2022)
- [Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation](https://mlsanity.com/paper/2303.09919/) (2023)
- [Visual Odometry with an Event Camera Using Continuous Ray Warping and Volumetric Contrast Maximization](https://mlsanity.com/paper/2107.03011/) (2021)
- [Globally-Optimal Event Camera Motion Estimation](https://mlsanity.com/paper/2203.03914/) (2022)
- [Automotive Object Detection via Learning Sparse Events by Spiking Neurons](https://mlsanity.com/paper/2307.12900/) (2023)
- [Exploiting Invariance in Training Deep Neural Networks](https://mlsanity.com/paper/2103.16634/) (2021)
- [Continuous-Time Human Motion Field from Events](https://mlsanity.com/paper/2412.01747/) (2024)

## Abstract (arXiv)

In this work we present a lightweight, unsupervised learning pipeline for textitdense depth, optical flow and egomotion estimation from sparse event output of the Dynamic Vision Sensor (DVS). To tackle this low level vision task, we use a novel encoder-decoder neural network architecture - ECN. Our work is the first monocular pipeline that generates dense depth and optical flow from sparse event data only. The network works in self-supervised mode and has just 150k parameters. We evaluate our pipeline on the MVSEC self driving dataset and present results for depth, optical flow and and egomotion estimation. Due to the lightweight design, the inference part of the network runs at 250 FPS on a single GPU, making the pipeline ready for realtime robotics applications. Our experiments demonstrate significant improvements upon previous works that used deep learning on event data, as well as the ability of our pipeline to perform well during both day and night.