# RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark

arXiv: https://arxiv.org/abs/2407.13930 · HTML: https://mlsanity.com/paper/2407.13930/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [Deep Learning-Based Human Pose Estimation: A Survey](https://mlsanity.com/paper/2012.13392/) (2020)
- [StableVideo: Text-driven Consistency-aware Diffusion Video Editing](https://mlsanity.com/paper/2308.09592/) (2023)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors](https://mlsanity.com/paper/2103.17265/) (2021)
- [SimPoE: Simulated Character Control for 3D Human Pose Estimation](https://mlsanity.com/paper/2104.00683/) (2021)
- [K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions](https://mlsanity.com/paper/2206.08171/) (2022)
- [mm-Pose: Real-Time Human Skeletal Posture Estimation using mmWave Radars and CNNs](https://mlsanity.com/paper/1911.09592/) (2019)
- [RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization](https://mlsanity.com/paper/2102.05150/) (2021)
- [SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments](https://mlsanity.com/paper/2303.09095/) (2023)
- [HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR](https://mlsanity.com/paper/2203.09215/) (2022)
- [mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors](https://mlsanity.com/paper/2210.08394/) (2022)
- [mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar](https://mlsanity.com/paper/2209.05070/) (2022)
- [HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar](https://mlsanity.com/paper/2210.12564/) (2022)
- [Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation](https://mlsanity.com/paper/2307.03833/) (2023)
- [Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation](https://mlsanity.com/paper/2303.16456/) (2023)
- [PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2308.09678/) (2023)
- [mmPose-NLP: A Natural Language Processing Approach to Precise Skeletal Pose Estimation using mmWave Radars](https://mlsanity.com/paper/2107.10327/) (2021)
- [Absolute Human Pose Estimation with Depth Prediction Network](https://mlsanity.com/paper/1904.05947/) (2019)
- [CenterRadarNet: Joint 3D Object Detection and Tracking Framework using 4D FMCW Radar](https://mlsanity.com/paper/2311.01423/) (2023)
- [Efficient Domain Adaptation via Generative Prior for 3D Infant Pose Estimation](https://mlsanity.com/paper/2311.12043/) (2023)
- [CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations](https://mlsanity.com/paper/2301.02979/) (2023)
- [MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling](https://mlsanity.com/paper/2306.17201/) (2023)
- [Vision meets mmWave Radar: 3D Object Perception Benchmark for Autonomous Driving](https://mlsanity.com/paper/2311.10261/) (2023)

## Built on by (most cited first, 16 of 16)

- [M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstruction](https://mlsanity.com/paper/2512.12378/) (2025)
- [RAPTR: Radar-based 3D Pose Estimation using Transformer](https://mlsanity.com/paper/2511.08387/) (2025)
- [RISE: Single Static Radar-based Indoor Scene Understanding](https://mlsanity.com/paper/2511.14019/) (2025)
- [milliMamba: Specular-Aware Human Pose Estimation via Dual mmWave Radar with Multi-Frame Mamba Fusion](https://mlsanity.com/paper/2512.20128/) (2025)
- [WiCompass: Oracle-driven Data Scaling for mmWave Human Pose Estimation](https://mlsanity.com/paper/2602.18726/) (2026)
- [Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasets](https://mlsanity.com/paper/2603.14507/) (2026)
- [DGHMesh: A Large-scale Dual-radar mmWave Dataset and Generalization-focused Benchmark for Human Mesh Reconstruction](https://mlsanity.com/paper/2604.22827/) (2026)
- [A Two-Stage Motion-Aware Framework for mmWave-based Human Mesh Recovery](https://mlsanity.com/paper/2605.08530/) (2026)
- [Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation](https://mlsanity.com/paper/2607.18875/) (2026)
- [mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar](https://mlsanity.com/paper/2609.34220/) (2026)
- [ToddlerAct: A Toddler Action Recognition Dataset for Gross Motor Development Assessment](https://mlsanity.com/paper/2409.00349/) (2024)
- [RadProPoser: Probabilistic Radar Tensor Human Pose Estimation That Knows Its Limits](https://mlsanity.com/paper/2508.03578/) (2025)
- [Privacy-Preserving Industrial Ergonomics: mmWave-Based Automated REBA Scoring and Pose Estimation](https://mlsanity.com/paper/2607.02611/) (2026)
- [mmSimPrior: Learning Simulation Priors for Data-Efficient and Generalizable Real-World Radar-based Human Motion Reconstruction](https://mlsanity.com/paper/2607.22973/) (2026)
- [Learning Biomechanically Plausible Human Motion from Sparse Radar Point Clouds](https://mlsanity.com/paper/2608.03637/) (2026)
- [You Only Flow Once: Calibrated and Real-Time Radar Pose Estimation with Multi-Hypothesis Normalizing Flows](https://mlsanity.com/paper/2608.09579/) (2026)

## Abstract (arXiv)

Traditional methods for human localization and pose estimation (HPE), which mainly rely on RGB images as an input modality, confront substantial limitations in real-world applications due to privacy concerns. In contrast, radar-based HPE methods emerge as a promising alternative, characterized by distinctive attributes such as through-wall recognition and privacy-preserving, rendering the method more conducive to practical deployments. This paper presents a Radar Tensor-based human pose (RT-Pose) dataset and an open-source benchmarking framework. The RT-Pose dataset comprises 4D radar tensors, LiDAR point clouds, and RGB images, and is collected for a total of 72k frames across 240 sequences with six different complexity-level actions. The 4D radar tensor provides raw spatio-temporal information, differentiating it from other radar point cloud-based datasets. We develop an annotation process using RGB images and LiDAR point clouds to accurately label 3D human skeletons. In addition, we propose HRRadarPose, the first single-stage architecture that extracts the high-resolution representation of 4D radar tensors in 3D space to aid human keypoint estimation. HRRadarPose outperforms previous radar-based HPE work on the RT-Pose benchmark. The overall HRRadarPose performance on the RT-Pose dataset, as reflected in a mean per joint position error (MPJPE) of 9.91cm, indicates the persistent challenges in achieving accurate HPE in complex real-world scenarios. RT-Pose is available at https://huggingface.co/datasets/uwipl/RT-Pose.