# DIR-BHRNet: A Lightweight Network for Real-time Vision-based Multi-person Pose Estimation on Smartphones

arXiv: https://arxiv.org/abs/2407.13777 · HTML: https://mlsanity.com/paper/2407.13777/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [DeepCache: Principled Cache for Mobile Deep Vision](https://mlsanity.com/paper/1712.01670/) (2017)
- [AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search](https://mlsanity.com/paper/2001.04246/) (2020)
- [Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose](https://mlsanity.com/paper/1811.12004/) (2018)
- [YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design](https://mlsanity.com/paper/2009.05697/) (2020)
- [Vision-Based Human Pose Estimation via Deep Learning: A Survey](https://mlsanity.com/paper/2308.13872/) (2023)
- [EfficientHRNet: Efficient Scaling for Lightweight High-Resolution Multi-Person Pose Estimation](https://mlsanity.com/paper/2007.08090/) (2020)
- [Towards Fast and Accurate Multi-Person Pose Estimation on Mobile Devices](https://mlsanity.com/paper/2106.15304/) (2021)
- [MDPose: Real-Time Multi-Person Pose Estimation via Mixture Density Model](https://mlsanity.com/paper/2302.08751/) (2023)

## Built on by (most cited first, 4 of 4)

- [ML-SceGen: A Multi-level Scenario Generation Framework](https://mlsanity.com/paper/2501.10782/) (2025)
- [Development of Occupancy Prediction Algorithm for Underground Parking Lots](https://mlsanity.com/paper/2409.00923/) (2024)
- [Manboformer: Learning Gaussian Representations via Spatial-temporal Attention Mechanism](https://mlsanity.com/paper/2503.04863/) (2025)
- [Data Shift of Object Detection in Autonomous Driving](https://mlsanity.com/paper/2508.11868/) (2025)

## Abstract (arXiv)

Human pose estimation (HPE), particularly multi-person pose estimation (MPPE), has been applied in many domains such as human-machine systems. However, the current MPPE methods generally run on powerful GPU systems and take a lot of computational costs. Real-time MPPE on mobile devices with low-performance computing is a challenging task. In this paper, we propose a lightweight neural network, DIR-BHRNet, for real-time MPPE on smartphones. In DIR-BHRNet, we design a novel lightweight convolutional module, Dense Inverted Residual (DIR), to improve accuracy by adding a depthwise convolution and a shortcut connection into the well-known Inverted Residual, and a novel efficient neural network structure, Balanced HRNet (BHRNet), to reduce computational costs by reconfiguring the proper number of convolutional blocks on each branch. We evaluate DIR-BHRNet on the well-known COCO and CrowdPose datasets. The results show that DIR-BHRNet outperforms the state-of-the-art methods in terms of accuracy with a real-time computational cost. Finally, we implement the DIR-BHRNet on the current mainstream Android smartphones, which perform more than 10 FPS. The free-used executable file (Android 10), source code, and a video description of this work are publicly available on the page 1 to facilitate the development of real-time MPPE on smartphones.