mlsanityData to 7 Oct 2026

A Spatio-Temporal Multilayer Perceptron for Gesture Recognition

arXiv 2204.11511 ↗First posted 25 Apr 2022cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Gesture recognition is essential for the interaction of autonomous vehicles with humans. While the current approaches focus on combining several modalities like image features, keypoints and bone vectors, we present neural network architecture that delivers state-of-the-art results only with body skeleton input data. We propose the spatio-temporal multilayer perceptron for gesture recognition in the context of autonomous vehicles. Given 3D body poses over time, we define temporal and spatial mixing operations to extract features in both domains. Additionally, the importance of each time step is re-weighted with Squeeze-and-Excitation layers. An extensive evaluation of the TCG and Drive&Act datasets is provided to showcase the promising performance of our approach. Furthermore, we deploy our model to our autonomous vehicle to show its real-time capability and stable execution.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 53 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

23 of 53 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2016Layer Normalizationcited by 5,579
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  11. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  12. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
11 more
  1. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  2. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  3. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333
  4. 2016A Decomposable Attention Model for Natural Language Inferencecited by 214
  5. 2019PifPaf: Composite Fields for Human Pose Estimationcited by 120
  6. 2018Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practicecited by 74
  7. 2020Traffic Control Gesture Recognition for Autonomous Vehiclescited by 24
  8. 2020Pedestrian Path, Pose and Intention Prediction through Gaussian Process Dynamical Models and Pedestrian Activity Recognitioncited by 20
  9. 2018Looking at the Driver/Rider in Autonomous Vehicles to Predict Take-Over Readinesscited by 13
  10. 2021Self-Supervised 3D Human Pose Estimation with Multiple-View Geometrycited by 10
  11. 2021Learning Temporal 3D Human Pose Estimation with Pseudo-Labelscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 1.

5 papers.

  1. 2024PO-GUISE+: Pose and object guided transformer token selection for efficient driver action recognitioncited by 2
  2. 2023Gesture Recognition with Keypoint and Radar Stream Fusion for Automated Vehiclescited by 1
  3. 2022Efficient Gesture Recognition for the Assistance of Visually Impaired People using Multi-Head Neural Networkscited by 0
  4. 2023Data-Free Backbone Fine-Tuning for Pruned Neural Networkscited by 0
  5. 2023Pedestrian Environment Model for Automated Drivingcited by 0