# Fast Trajectory End-Point Prediction with Event Cameras for Reactive Robot Control

arXiv: https://arxiv.org/abs/2302.13796 · HTML: https://mlsanity.com/paper/2302.13796/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 7 of 7)

- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [FAST-Dynamic-Vision: Detection and Tracking Dynamic Objects with Event and Depth Sensing](https://mlsanity.com/paper/2103.05903/) (2021)
- [Catch the Ball: Accurate High-Speed Motions for Mobile Manipulators via Inverse Dynamics Learning](https://mlsanity.com/paper/2003.07489/) (2020)
- [Real-Time Grasping Strategies Using Event Camera](https://mlsanity.com/paper/2107.07200/) (2021)
- [Exploiting Event Cameras for Spatio-Temporal Prediction of Fast-Changing Trajectories](https://mlsanity.com/paper/2001.01248/) (2020)

## Built on by (most cited first, 6 of 6)

- [SEVD: Synthetic Event-based Vision Dataset for Ego and Fixed Traffic Perception](https://mlsanity.com/paper/2404.10540/) (2024)
- [Drone Detection with Event Cameras](https://mlsanity.com/paper/2508.04564/) (2025)
- [Towards Low-Latency Event-based Obstacle Avoidance on a FPGA-Drone](https://mlsanity.com/paper/2504.10400/) (2025)
- [Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction](https://mlsanity.com/paper/2506.07860/) (2025)
- [RGB-Event Fusion with Self-Attention for Collision Prediction](https://mlsanity.com/paper/2505.04258/) (2025)
- [Learning from Noise: Enhancing DNNs for Event-Based Vision through Controlled Noise Injection](https://mlsanity.com/paper/2506.03918/) (2025)

## Abstract (arXiv)

Prediction skills can be crucial for the success of tasks where robots have limited time to act or joints actuation power. In such a scenario, a vision system with a fixed, possibly too low, sampling rate could lead to the loss of informative points, slowing down prediction convergence and reducing the accuracy. In this paper, we propose to exploit the low latency, motion-driven sampling, and data compression properties of event cameras to overcome these issues. As a use-case, we use a Panda robotic arm to intercept a ball bouncing on a table. To predict the interception point, we adopt a Stateful LSTM network, a specific LSTM variant without fixed input length, which perfectly suits the event-driven paradigm and the problem at hand, where the length of the trajectory is not defined. We train the network in simulation to speed up the dataset acquisition and then fine-tune the models on real trajectories. Experimental results demonstrate how using a dense spatial sampling (i.e. event cameras) significantly increases the number of intercepted trajectories as compared to a fixed temporal sampling (i.e. frame-based cameras).