# Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds

arXiv: https://arxiv.org/abs/2109.06448 · HTML: https://mlsanity.com/paper/2109.06448/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 71 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition with Multimodal Training](https://mlsanity.com/paper/1812.06145/) (2018)
- [Making the Invisible Visible: Action Recognition Through Walls and Occlusions](https://mlsanity.com/paper/1909.09300/) (2019)
- [WiGest: A Ubiquitous WiFi-based Gesture Recognition System](https://mlsanity.com/paper/1501.04301/) (2015)
- [Spatiotemporal Learning of Dynamic Gestures from 3D Point Cloud Data](https://mlsanity.com/paper/1804.08859/) (2018)

## Built on by (most cited first, 10 of 10)

- [Large Model for Small Data: Foundation Model for Cross-Modal RF Human Activity Recognition](https://mlsanity.com/paper/2410.19766/) (2024)
- [GesturePrint: Enabling User Identification for mmWave-based Gesture Recognition Systems](https://mlsanity.com/paper/2408.05358/) (2024)
- [Cross-modal Learning of Graph Representations using Radar Point Cloud for Long-Range Gesture Recognition](https://mlsanity.com/paper/2203.17066/) (2022)
- [mmSpyVR: Exploiting mmWave Radar for Penetrating Obstacles to Uncover Privacy Vulnerability of Virtual Reality](https://mlsanity.com/paper/2411.09914/) (2024)
- [User Localization using RF Sensing: A Performance comparison between LIS and mmWave Radars](https://mlsanity.com/paper/2205.10321/) (2022)
- [Cooperation and Federation in Distributed Radar Point Cloud Processing](https://mlsanity.com/paper/2405.01995/) (2024)
- [Environment-Aware Transfer Reinforcement Learning for Sustainable Beam Selection](https://mlsanity.com/paper/2511.11647/) (2025)
- [Exploring Spatial-Temporal Representation via Star Graph for mmWave Radar-based Human Activity Recognition](https://mlsanity.com/paper/2512.12013/) (2025)
- [PCFEx: Point Cloud Feature Extraction for Graph Neural Networks](https://mlsanity.com/paper/2603.08540/) (2026)
- [DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognition](https://mlsanity.com/paper/2605.09604/) (2026)

## Abstract (arXiv)

We present Tesla-Rapture, a gesture recognition interface for point clouds generated by mmWave Radars. State of the art gesture recognition models are either too resource consuming or not sufficiently accurate for integration into real-life scenarios using wearable or constrained equipment such as IoT devices (e.g. Raspberry PI), XR hardware (e.g. HoloLens), or smart-phones. To tackle this issue, we developed Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. The model outperforms the state of the art on two datasets in terms of accuracy while reducing the computational complexity and, hence, the execution time. In particular, the approach, is able to predict a gesture almost 8 times faster than the most accurate competitor. Our performance evaluation in different scenarios (environments, angles, distances) shows that Tesla generalizes well and improves the accuracy up to 20% in challenging scenarios like a through-wall setting and sensing at extreme angles. Utilizing Tesla, we develop Tesla-Rapture, a real-time implementation using a mmWave Radar on a Raspberry PI 4 and evaluate its accuracy and time-complexity. We also publish the source code, the trained models, and the implementation of the model for embedded devices.