mlsanityData to 7 Oct 2026

N-DriverMotion: Driver motion learning and prediction using an event-based camera and directly trained spiking neural networks on Loihi 2

arXiv 2408.13379 ↗First posted 23 Aug 2024cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Driver motion recognition is a principal factor in ensuring the safety of driving systems. This paper presents a novel system for learning and predicting driver motions and an event-based high-resolution (1280x720) dataset, N-DriverMotion, newly collected to train on a neuromorphic vision system. The system comprises an event-based camera that generates the first high-resolution driver motion dataset representing spike inputs and efficient spiking neural networks (SNNs) that are effective in training and predicting the driver's gestures. The event dataset consists of 13 driver motion categories classified by direction (front, side), illumination (bright, moderate, dark), and participant. A novel simplified four-layer convolutional spiking neural network (CSNN) that we proposed was directly trained using the high-resolution dataset without any time-consuming preprocessing. This enables efficient adaptation to on-device SNNs for real-time inference on high-resolution event-based streams. Compared with recent gesture recognition systems adopting neural networks for vision processing, the proposed neuromorphic vision system achieves comparable accuracy, 94.04%, in recognizing driver motions with the CSNN architecture. Our proposed CSNN and the dataset can be used to develop safer and more efficient driver monitoring systems for autonomous vehicles or edge devices requiring an efficient neural network architecture.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

19 of 40 references.

References · 19

  1. 2019Event-based Vision: A Surveycited by 1,010
  2. 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 308
  3. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 257
  4. 2019End-to-End Learning of Representations for Asynchronous Event-Based Datacited by 216
  5. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 203
  6. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 197
  7. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
  8. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
  9. 2020Event-based Asynchronous Sparse Convolutional Networkscited by 103
  10. 2021Efficient Neuromorphic Signal Processing with Loihi 2cited by 90
  11. 2023Deep Directly-Trained Spiking Neural Networks for Object Detectioncited by 76
  12. 2021Time-Ordered Recent Event (TORE) Volumes for Event Camerascited by 73
7 more
  1. 2018Benchmarking Keyword Spotting Efficiency on Neuromorphic Hardwarecited by 37
  2. 2023Efficient Video and Audio processing with Loihi 2cited by 28
  3. 2021CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processorcited by 27
  4. 2020An Efficient Spiking Neural Network for Recognizing Gestures with a DVS Camera on the Loihi Neuromorphic Processorcited by 25
  5. 2016Unsupervised Regenerative Learning of Hierarchical Features in Spiking Deep Networks for Object Recognitioncited by 6
  6. 2022Impact of spiking neurons leakages and network recurrences on event-based spatio-temporal pattern recognitioncited by 4
  7. 2023Implementing and Benchmarking the Locally Competitive Algorithm on the Loihi 2 Neuromorphic Processorcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DeepIPCv3: Event-Aware Multi-Modal Sensor Fusion for Sudden Pedestrian Crossing Avoidancecited by 0