mlsanityData to 7 Oct 2026

Representation Learning for Event-based Visuomotor Policies

arXiv 2103.00806 ↗First posted 1 Mar 2021cs.CV cs.ROcited by 19

Not on the map. Why?

Abstract (arXiv)

Event-based cameras are dynamic vision sensors that provide asynchronous measurements of changes in per-pixel brightness at a microsecond level. This makes them significantly faster than conventional frame-based cameras, and an appealing choice for high-speed navigation. While an interesting sensor modality, this asynchronously streamed event data poses a challenge for machine learning techniques that are more suited for frame-based data. In this paper, we present an event variational autoencoder and show that it is feasible to learn compact representations directly from asynchronous spatiotemporal event data. Furthermore, we show that such pretrained representations can be used for event-based reinforcement learning instead of end-to-end reward driven perception. We validate this framework of learning event-based visuomotor policies by applying it to an obstacle avoidance scenario in simulation. Compared to techniques that treat event data as images, we show that representations learnt from event streams result in faster policy training, adapt to different control capacities, and demonstrate a higher degree of robustness.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 1 Mar 2021.

Builds on

29 of 50 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2018World Modelscited by 1,444
  7. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  9. 2015Generating Sentences from a Continuous Spacecited by 607
  10. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  11. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  12. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
17 more
  1. 2018EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Camerascited by 223
  2. 2019End-to-End Learning of Representations for Asynchronous Event-Based Datacited by 216
  3. 2018HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classificationcited by 215
  4. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 197
  5. 2020Event-based Asynchronous Sparse Convolutional Networkscited by 103
  6. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 98
  7. 2019Does computer vision matter for action?cited by 64
  8. 2018EventNet: Asynchronous Recursive Event Processingcited by 58
  9. 2018Asynchronous Convolutional Networks for Object Detection in Neuromorphic Camerascited by 57
  10. 2016Neuromorphic Deep Learning Machinescited by 38
  11. 2020Event Probability Mask (EPM) and Event Denoising Convolutional Neural Network (EDnCNN) for Neuromorphic Camerascited by 38
  12. 2020Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardwarecited by 32
  13. 2019Towards Low-Latency High-Bandwidth Control of Quadrotors using Event Camerascited by 14
  14. 2017Vertical Landing for Micro Air Vehicles using Event-Based Optical Flowcited by 12
  15. 2018Playing Atari with Six Neuronscited by 11
  16. 2020Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVscited by 7
  17. 2020Exploration of Reinforcement Learning for Event Camera using Car-like Robotscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 5; 2025: 4; 2026: 4.

19 papers.

Sort:

By area:

  1. 2022Event Transformer. A sparse-aware solution for efficient event data processingcited by 48
  2. 2022Event Transformer+. A multi-purpose solution for efficient event data processingcited by 13
  3. 2024Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithmscited by 11
  4. 2023Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Cameracited by 5
  5. 2025SEBVS: Synthetic Event-based Visual Servoing for Robot Navigation and Manipulationcited by 5
  6. 2023Point-Voxel Absorbing Graph Representation Learning for Event Stream based Recognitioncited by 4
  7. 2024Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognitioncited by 3
  8. 2025Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Reviewcited by 3
  9. 2026EventFlash: Towards Efficient MLLMs for Event-Based Visioncited by 3
  10. 2024Text-to-Events: Synthetic Event Camera Streams from Conditional Text Inputcited by 2
  11. 2025Learning to Remove Lens Flare in Event Cameracited by 2
  12. 2023CERiL: Continuous Event-based Reinforcement Learningcited by 1
7 more
  1. 2024Uncertainty-aware Bridge based Mobile-Former Network for Event-based Pattern Recognitioncited by 1
  2. 2024Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Trackingcited by 1
  3. 2023Event Fusion Photometric Stereo Networkcited by 0
  4. 2025Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement Learningcited by 0
  5. 2026CADENCE: Context-Adaptive Depth Estimation for Navigation and Computational Efficiencycited by 0
  6. 2026FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learningcited by 0
  7. 2026ECHO: Event-Augmented Context with Hindsight and Outlook for Wrist-Only Manipulationcited by 0