# A multi-modal table tennis robot system

arXiv: https://arxiv.org/abs/2310.19062 · HTML: https://mlsanity.com/paper/2310.19062/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Learning to Play Table Tennis From Scratch using Muscular Robots](https://mlsanity.com/paper/2006.05935/) (2020)
- [Robotic Table Tennis with Model-Free Reinforcement Learning](https://mlsanity.com/paper/2003.14398/) (2020)
- [Robotic Table Tennis: A Case Study into a High Speed Learning System](https://mlsanity.com/paper/2309.03315/) (2023)
- [Spin Detection in Robotic Table Tennis](https://mlsanity.com/paper/1905.07967/) (2019)
- [GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot](https://mlsanity.com/paper/2210.03662/) (2022)
- [SpinDOE: A ball spin estimation method for table tennis robot](https://mlsanity.com/paper/2303.03879/) (2023)
- [eWand: A calibration framework for wide baseline frame-based and event-based camera systems](https://mlsanity.com/paper/2309.12685/) (2023)
- [Reliable Real Time Ball Tracking for Robot Table Tennis](https://mlsanity.com/paper/1908.07332/) (2019)

## Built on by (most cited first, 1 of 1)

- [Detection of Fast-Moving Objects with Neuromorphic Hardware](https://mlsanity.com/paper/2403.10677/) (2024)

## Abstract (arXiv)

In recent years, robotic table tennis has become a popular research challenge for perception and robot control. Here, we present an improved table tennis robot system with high accuracy vision detection and fast robot reaction. Based on previous work, our system contains a KUKA robot arm with 6 DOF, with four frame-based cameras and two additional event-based cameras. We developed a novel calibration approach to calibrate this multimodal perception system. For table tennis, spin estimation is crucial. Therefore, we introduced a novel, and more accurate spin estimation approach. Finally, we show how combining the output of an event-based camera and a Spiking Neural Network (SNN) can be used for accurate ball detection.