# Traffic Control Gesture Recognition for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2007.16072 · HTML: https://mlsanity.com/paper/2007.16072/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 24 of 24)

- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network](https://mlsanity.com/paper/2310.16288/) (2023)
- [MotionMixer: MLP-based 3D Human Body Pose Forecasting](https://mlsanity.com/paper/2207.00499/) (2022)
- [Anomaly Detection in Multi-Agent Trajectories for Automated Driving](https://mlsanity.com/paper/2110.07922/) (2021)
- [Lightweight Monocular Depth Estimation through Guided Decoding](https://mlsanity.com/paper/2203.04206/) (2022)
- [Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry](https://mlsanity.com/paper/2108.07777/) (2021)
- [Can Vision-Language Models Understand and Interpret Dynamic Gestures from Pedestrians? Pilot Datasets and Exploration Towards Instructive Nonverbal Commands for Cooperative Autonomous Vehicles](https://mlsanity.com/paper/2504.10873/) (2025)
- [Weakly Supervised Multi-Modal 3D Human Body Pose Estimation for Autonomous Driving](https://mlsanity.com/paper/2307.14889/) (2023)
- [Extrinsic Camera Calibration with Semantic Segmentation](https://mlsanity.com/paper/2208.03949/) (2022)
- [A Spatio-Temporal Multilayer Perceptron for Gesture Recognition](https://mlsanity.com/paper/2204.11511/) (2022)
- [Learning Temporal 3D Human Pose Estimation with Pseudo-Labels](https://mlsanity.com/paper/2110.07578/) (2021)
- [DeFeeNet: Consecutive 3D Human Motion Prediction with Deviation Feedback](https://mlsanity.com/paper/2304.04496/) (2023)
- [Optimal Solving of Constrained Path-Planning Problems with Graph Convolutional Networks and Optimized Tree Search](https://mlsanity.com/paper/2108.01036/) (2021)
- [Visual Domain Adaptation for Monocular Depth Estimation on Resource-Constrained Hardware](https://mlsanity.com/paper/2108.02671/) (2021)
- [Language-Assisted Human Part Motion Learning for Skeleton-Based Temporal Action Segmentation](https://mlsanity.com/paper/2410.06353/) (2024)
- [Mamba-Driven Topology Fusion for Monocular 3D Human Pose Estimation](https://mlsanity.com/paper/2505.20611/) (2025)
- [Attention-based Vehicle Self-Localization with HD Feature Maps](https://mlsanity.com/paper/2107.07787/) (2021)
- [Gesture Recognition with Keypoint and Radar Stream Fusion for Automated Vehicles](https://mlsanity.com/paper/2302.09998/) (2023)
- [Spectral Compression Transformer with Line Pose Graph for Monocular 3D Human Pose Estimation](https://mlsanity.com/paper/2505.21309/) (2025)
- [How do Foundation Models Compare to Skeleton-Based Approaches for Gesture Recognition in Human-Robot Interaction?](https://mlsanity.com/paper/2506.20795/) (2025)
- [Pedestrian Environment Model for Automated Driving](https://mlsanity.com/paper/2308.09080/) (2023)
- [Gesture Matters: Pedestrian Gesture Recognition for AVs Through Skeleton Pose Evaluation](https://mlsanity.com/paper/2602.08479/) (2026)
- [LaDy: Lagrangian-Dynamic Informed Network for Skeleton-based Action Segmentation via Spatial-Temporal Modulation](https://mlsanity.com/paper/2603.24097/) (2026)
- [Spectral Scalpel: Amplifying Adjacent Action Discrepancy via Frequency-Selective Filtering for Skeleton-Based Action Segmentation](https://mlsanity.com/paper/2603.24134/) (2026)

## Abstract (arXiv)

A car driver knows how to react on the gestures of the traffic officers. Clearly, this is not the case for the autonomous vehicle, unless it has road traffic control gesture recognition functionalities. In this work, we address the limitation of the existing autonomous driving datasets to provide learning data for traffic control gesture recognition. We introduce a dataset that is based on 3D body skeleton input to perform traffic control gesture classification on every time step. Our dataset consists of 250 sequences from several actors, ranging from 16 to 90 seconds per sequence. To evaluate our dataset, we propose eight sequential processing models based on deep neural networks such as recurrent networks, attention mechanism, temporal convolutional networks and graph convolutional networks. We present an extensive evaluation and analysis of all approaches for our dataset, as well as real-world quantitative evaluation. The code and dataset is publicly available.