# Spatiotemporal Learning of Dynamic Gestures from 3D Point Cloud Data

arXiv: https://arxiv.org/abs/1804.08859 · HTML: https://mlsanity.com/paper/1804.08859/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2018.

## Built on by (most cited first, 3 of 3)

- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds](https://mlsanity.com/paper/2109.06448/) (2021)
- [Integrating Sensing and Communication in Cellular Networks via NR Sidelink](https://mlsanity.com/paper/2109.07253/) (2021)

## Abstract (arXiv)

In this paper, we demonstrate an end-to-end spatiotemporal gesture learning approach for 3D point cloud data using a new gestures dataset of point clouds acquired from a 3D sensor. Nine classes of gestures were learned from gestures sample data. We mapped point cloud data into dense occupancy grids, then time steps of the occupancy grids are used as inputs into a 3D convolutional neural network which learns the spatiotemporal features in the data without explicit modeling of gesture dynamics. We also introduced a 3D region of interest jittering approach for point cloud data augmentation. This resulted in an increased classification accuracy of up to 10% when the augmented data is added to the original training data. The developed model is able to classify gestures from the dataset with 84.44% accuracy. We propose that point cloud data will be a more viable data type for scene understanding and motion recognition, as 3D sensors become ubiquitous in years to come.