# RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation

arXiv: https://arxiv.org/abs/2312.07526 · HTML: https://mlsanity.com/paper/2312.07526/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 52 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [VarifocalNet: An IoU-aware Dense Object Detector](https://mlsanity.com/paper/2008.13367/) (2020)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [RTMDet: An Empirical Study of Designing Real-Time Object Detectors](https://mlsanity.com/paper/2212.07784/) (2022)
- [Transformer Quality in Linear Time](https://mlsanity.com/paper/2202.10447/) (2022)
- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [PoseTrack: A Benchmark for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1710.10000/) (2017)
- [Bounding Box Regression with Uncertainty for Accurate Object Detection](https://mlsanity.com/paper/1809.08545/) (2018)
- [Data Uncertainty Learning in Face Recognition](https://mlsanity.com/paper/2003.11339/) (2020)
- [Human Pose Regression with Residual Log-likelihood Estimation](https://mlsanity.com/paper/2107.11291/) (2021)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression](https://mlsanity.com/paper/2104.02300/) (2021)
- [TokenPose: Learning Keypoint Tokens for Human Pose Estimation](https://mlsanity.com/paper/2104.03516/) (2021)
- [Single-Stage Multi-Person Pose Machines](https://mlsanity.com/paper/1908.09220/) (2019)
- [AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding](https://mlsanity.com/paper/1711.06475/) (2017)

## Built on by (most cited first, 30 of 34)

- [RTMW: Real-Time Multi-Person 2D and 3D Whole-body Pose Estimation](https://mlsanity.com/paper/2407.08634/) (2024)
- [The BabyView dataset: High-resolution egocentric videos of infants' and young children's everyday experiences](https://mlsanity.com/paper/2406.10447/) (2024)
- [ProbPose: A Probabilistic Approach to 2D Human Pose Estimation](https://mlsanity.com/paper/2412.02254/) (2024)
- [EDM: Efficient Deep Feature Matching](https://mlsanity.com/paper/2503.05122/) (2025)
- [Referring Human Pose and Mask Estimation in the Wild](https://mlsanity.com/paper/2410.20508/) (2024)
- [Group Inertial Poser: Multi-Person Pose and Global Translation from Sparse Inertial Sensors and Ultra-Wideband Ranging](https://mlsanity.com/paper/2510.21654/) (2025)
- [EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation](https://mlsanity.com/paper/2603.18739/) (2026)
- [Pose-Aware Weakly-Supervised Action Segmentation](https://mlsanity.com/paper/2504.05700/) (2025)
- [MamKPD: A Simple Mamba Baseline for Real-Time 2D Keypoint Detection](https://mlsanity.com/paper/2412.01422/) (2024)
- [Differentially Private 2D Human Pose Estimation](https://mlsanity.com/paper/2504.10190/) (2025)
- [Unsupervised Cross-Domain 3D Human Pose Estimation via Pseudo-Label-Guided Global Transforms](https://mlsanity.com/paper/2504.12699/) (2025)
- [DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints](https://mlsanity.com/paper/2506.13027/) (2025)
- [DiffPose-Animal: A Language-Conditioned Diffusion Framework for Animal Pose Estimation](https://mlsanity.com/paper/2508.08783/) (2025)
- [CompressAI-Vision: Open-source software to evaluate compression methods for computer vision tasks](https://mlsanity.com/paper/2509.20777/) (2025)
- [End-to-End Multi-Person Pose Estimation with Pose-Aware Video Transformer](https://mlsanity.com/paper/2511.13208/) (2025)
- [MultiCam: On-the-fly Multi-Camera Pose Estimation Using Spatiotemporal Overlaps of Known Objects](https://mlsanity.com/paper/2603.22839/) (2026)
- [From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models](https://mlsanity.com/paper/2607.06553/) (2026)
- [Estimating Human Poses Across Datasets: A Unified Skeleton and Multi-Teacher Distillation Approach](https://mlsanity.com/paper/2405.20084/) (2024)
- [Human Pose-Constrained UV Map Estimation](https://mlsanity.com/paper/2501.08815/) (2025)
- [Keypoints as Dynamic Centroids for Unified Human Pose and Segmentation](https://mlsanity.com/paper/2505.12130/) (2025)
- [SteerPose: Simultaneous Extrinsic Camera Calibration and Matching from Articulation](https://mlsanity.com/paper/2506.01691/) (2025)
- [Retrieval Augmented Comic Image Generation](https://mlsanity.com/paper/2506.12517/) (2025)
- [Parse Graph-Based Visual-Language Interaction for Human Pose Estimation](https://mlsanity.com/paper/2509.07385/) (2025)
- [Lightweight Facial Landmark Detection in Thermal Images via Multi-Level Cross-Modal Knowledge Transfer](https://mlsanity.com/paper/2510.11128/) (2025)
- [FlyPose: Towards Robust Human Pose Estimation From Aerial Views](https://mlsanity.com/paper/2601.05747/) (2026)
- [ER-Pose: Rethinking Keypoint-Driven Representation Learning for Real-Time Human Pose Estimation](https://mlsanity.com/paper/2603.08681/) (2026)
- [Multi-Person Pose Estimation Evaluation Using Optimal Transportation and Improved Pose Matching](https://mlsanity.com/paper/2603.10398/) (2026)
- [FSMC-Pose: Frequency and Spatial Fusion with Multiscale Self-calibration for Cattle Mounting Pose Estimation](https://mlsanity.com/paper/2603.16596/) (2026)
- [TouchMap-OR: Multi-View 3D Mapping of Hand-Surface Contacts](https://mlsanity.com/paper/2605.17638/) (2026)
- [Multi-Camera AR Guidance System for Surgical Instrument Handling and Assembly: Investigating Workload and Efficiency](https://mlsanity.com/paper/2606.04992/) (2026)

## Abstract (arXiv)

Real-time multi-person pose estimation presents significant challenges in balancing speed and precision. While two-stage top-down methods slow down as the number of people in the image increases, existing one-stage methods often fail to simultaneously deliver high accuracy and real-time performance. This paper introduces RTMO, a one-stage pose estimation framework that seamlessly integrates coordinate classification by representing keypoints using dual 1-D heatmaps within the YOLO architecture, achieving accuracy comparable to top-down methods while maintaining high speed. We propose a dynamic coordinate classifier and a tailored loss function for heatmap learning, specifically designed to address the incompatibilities between coordinate classification and dense prediction models. RTMO outperforms state-of-the-art one-stage pose estimators, achieving 1.1% higher AP on COCO while operating about 9 times faster with the same backbone. Our largest model, RTMO-l, attains 74.8% AP on COCO val2017 and 141 FPS on a single V100 GPU, demonstrating its efficiency and accuracy. The code and models are available at https://github.com/open-mmlab/mmpose/tree/main/projects/rtmo.