# mmPose-NLP: A Natural Language Processing Approach to Precise Skeletal Pose Estimation using mmWave Radars

arXiv: https://arxiv.org/abs/2107.10327 · HTML: https://mlsanity.com/paper/2107.10327/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 47 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation](https://mlsanity.com/paper/1511.06645/) (2015)
- [DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model](https://mlsanity.com/paper/1605.03170/) (2016)
- [mm-Pose: Real-Time Human Skeletal Posture Estimation using mmWave Radars and CNNs](https://mlsanity.com/paper/1911.09592/) (2019)
- [Scalable Sentiment for Sequence-to-sequence Chatbot Response with Performance Analysis](https://mlsanity.com/paper/1804.02504/) (2018)
- [Multiple Patients Behavior Detection in Real-time using mmWave Radar and Deep CNNs](https://mlsanity.com/paper/1911.06363/) (2019)

## Built on by (most cited first, 11 of 11)

- [HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar](https://mlsanity.com/paper/2210.12564/) (2022)
- [RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark](https://mlsanity.com/paper/2407.13930/) (2024)
- [MiliPoint: A Point Cloud Dataset for mmWave Radar](https://mlsanity.com/paper/2309.13425/) (2023)
- [milliMamba: Specular-Aware Human Pose Estimation via Dual mmWave Radar with Multi-Frame Mamba Fusion](https://mlsanity.com/paper/2512.20128/) (2025)
- [mmID: High-Resolution mmWave Imaging for Human Identification](https://mlsanity.com/paper/2402.00996/) (2024)
- [Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasets](https://mlsanity.com/paper/2603.14507/) (2026)
- [Skeleton Detection Using Dual Radars with Integration of Dual-View CNN Models and mmPose](https://mlsanity.com/paper/2411.19251/) (2024)
- [Exploring Spatial-Temporal Representation via Star Graph for mmWave Radar-based Human Activity Recognition](https://mlsanity.com/paper/2512.12013/) (2025)
- [mmGAT: Pose Estimation by Graph Attention with Mutual Features from mmWave Radar Point Cloud](https://mlsanity.com/paper/2603.08551/) (2026)
- [DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognition](https://mlsanity.com/paper/2605.09604/) (2026)
- [Learning Biomechanically Plausible Human Motion from Sparse Radar Point Clouds](https://mlsanity.com/paper/2608.03637/) (2026)

## Abstract (arXiv)

In this paper we presented mmPose-NLP, a novel Natural Language Processing (NLP) inspired Sequence-to-Sequence (Seq2Seq) skeletal key-point estimator using millimeter-wave (mmWave) radar data. To the best of the author's knowledge, this is the first method to precisely estimate upto 25 skeletal key-points using mmWave radar data alone. Skeletal pose estimation is critical in several applications ranging from autonomous vehicles, traffic monitoring, patient monitoring, gait analysis, to defense security forensics, and aid both preventative and actionable decision making. The use of mmWave radars for this task, over traditionally employed optical sensors, provide several advantages, primarily its operational robustness to scene lighting and adverse weather conditions, where optical sensor performance degrade significantly. The mmWave radar point-cloud (PCL) data is first voxelized (analogous to tokenization in NLP) and N frames of the voxelized radar data (analogous to a text paragraph in NLP) is subjected to the proposed mmPose-NLP architecture, where the voxel indices of the 25 skeletal key-points (analogous to keyword extraction in NLP) are predicted. The voxel indices are converted back to real world 3-D coordinates using the voxel dictionary used during the tokenization process. Mean Absolute Error (MAE) metrics were used to measure the accuracy of the proposed system against the ground truth, with the proposed mmPose-NLP offering <3 cm localization errors in the depth, horizontal and vertical axes. The effect of the number of input frames vs performance/accuracy was also studied for N = 1,2,..,10. A comprehensive methodology, results, discussions and limitations are presented in this paper. All the source codes and results are made available on GitHub for furthering research and development in this critical yet emerging domain of skeletal key-point estimation using mmWave radars.