# Steering a Predator Robot using a Mixed Frame/Event-Driven Convolutional Neural Network

arXiv: https://arxiv.org/abs/1606.09433 · HTML: https://mlsanity.com/paper/1606.09433/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2016.

## Built on by (most cited first, 15 of 15)

- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [GET: Group Event Transformer for Event-Based Vision](https://mlsanity.com/paper/2310.02642/) (2023)
- [Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation](https://mlsanity.com/paper/2109.04871/) (2021)
- [EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation](https://mlsanity.com/paper/2111.12341/) (2021)
- [Scene Adaptive Sparse Transformer for Event-based Object Detection](https://mlsanity.com/paper/2404.01882/) (2024)
- [Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation](https://mlsanity.com/paper/2109.01801/) (2021)
- [EVIMO2: An Event Camera Dataset for Motion Segmentation, Optical Flow, Structure from Motion, and Visual Inertial Odometry in Indoor Scenes with Monocular or Stereo Algorithms](https://mlsanity.com/paper/2205.03467/) (2022)
- [eTraM: Event-based Traffic Monitoring Dataset](https://mlsanity.com/paper/2403.19976/) (2024)
- [Emergent Visual Sensors for Autonomous Vehicles](https://mlsanity.com/paper/2205.09383/) (2022)
- [DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognition](https://mlsanity.com/paper/2407.05106/) (2024)
- [Event Camera Based Real-Time Detection and Tracking of Indoor Ground Robots](https://mlsanity.com/paper/2102.11916/) (2021)
- [Learning Vision-based Pursuit-Evasion Robot Policies](https://mlsanity.com/paper/2308.16185/) (2023)
- [Automotive Object Detection via Learning Sparse Events by Spiking Neurons](https://mlsanity.com/paper/2307.12900/) (2023)
- [Scalable Event-Based Video Streaming for Machines with MoQ](https://mlsanity.com/paper/2508.15003/) (2025)
- [Towards Closing the Domain Gap with Event Cameras](https://mlsanity.com/paper/2512.16178/) (2025)

## Abstract (arXiv)

This paper describes the application of a Convolutional Neural Network (CNN) in the context of a predator/prey scenario. The CNN is trained and run on data from a Dynamic and Active Pixel Sensor (DAVIS) mounted on a Summit XL robot (the predator), which follows another one (the prey). The CNN is driven by both conventional image frames and dynamic vision sensor "frames" that consist of a constant number of DAVIS ON and OFF events. The network is thus "data driven" at a sample rate proportional to the scene activity, so the effective sample rate varies from 15 Hz to 240 Hz depending on the robot speeds. The network generates four outputs: steer right, left, center and non-visible. After off-line training on labeled data, the network is imported on the on-board Summit XL robot which runs jAER and receives steering directions in real time. Successful results on closed-loop trials, with accuracies up to 87% or 92% (depending on evaluation criteria) are reported. Although the proposed approach discards the precise DAVIS event timing, it offers the significant advantage of compatibility with conventional deep learning technology without giving up the advantage of data-driven computing.