# SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens

arXiv: https://arxiv.org/abs/2411.19824 · HTML: https://mlsanity.com/paper/2411.19824/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 62 references that are in the map.
- First posted on arXiv 29 Nov 2024.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [AdaViT: Adaptive Tokens for Efficient Vision Transformer](https://mlsanity.com/paper/2112.07658/) (2021)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop](https://mlsanity.com/paper/2103.16507/) (2021)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [AdaViT: Adaptive Vision Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2111.15668/) (2021)
- [Monocular, One-stage, Regression of Multiple 3D People](https://mlsanity.com/paper/2008.12272/) (2020)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021)
- [Coherent Reconstruction of Multiple Humans from a Single Image](https://mlsanity.com/paper/2006.08586/) (2020)
- [QuadTree Attention for Vision Transformers](https://mlsanity.com/paper/2201.02767/) (2022)

## Built on by (most cited first, 16 of 16)

- [Human Mesh Modeling for Anny Body](https://mlsanity.com/paper/2511.03589/) (2025)
- [Syn4D: A Multiview Synthetic 4D Dataset](https://mlsanity.com/paper/2605.05207/) (2026)
- [Towards Metric-Aware Multi-Person Mesh Recovery by Jointly Optimizing Human Crowd in Camera Space](https://mlsanity.com/paper/2511.13282/) (2025)
- [NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data](https://mlsanity.com/paper/2603.18496/) (2026)
- [Phase-Aligned RoPE for Mixed-Resolution Diffusion Transformer](https://mlsanity.com/paper/2511.19778/) (2025)
- [OA-NBV: Occlusion-Aware Next-Best-View Planning for Human-Centered Active Perception on Mobile Robots](https://mlsanity.com/paper/2603.11072/) (2026)
- [SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild](https://mlsanity.com/paper/2605.07604/) (2026)
- [Multi-HMR 2: Multi-Person Camera-Centric Human Detection, Mesh Recovery and Tracking](https://mlsanity.com/paper/2606.14841/) (2026)
- [TAIHRI: Task-Aware 3D Human Keypoints Localization for Close-Range Human-Robot Interaction](https://mlsanity.com/paper/2604.08921/) (2026)
- [Anny-Fit: All-Age Human Mesh Recovery](https://mlsanity.com/paper/2605.04728/) (2026)
- [DETRAM: End-to-end DEtection, Tracking and Recovery of HumAn Meshes](https://mlsanity.com/paper/2607.09089/) (2026)
- [On the Use of Hierarchical Vision Foundation Models for Low-Cost Human Mesh Recovery and Pose Estimation](https://mlsanity.com/paper/2510.12660/) (2025)
- [SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance](https://mlsanity.com/paper/2604.13581/) (2026)
- [InterMesh: Explicit Interaction-Aware End-to-End Multi-Person Human Mesh Recovery](https://mlsanity.com/paper/2605.04554/) (2026)
- [Contrastive Multi-Modal Hypergraph Reasoning for 3D Crowd Mesh Recovery](https://mlsanity.com/paper/2605.13854/) (2026)
- [Robust Real-Time Monocular Human Pose Estimation with Insole-Based Sensor Fusion](https://mlsanity.com/paper/2610.19494/) (2026)

## Abstract (arXiv)

We propose a one-stage framework for real-time multi-person 3D human mesh estimation from a single RGB image. While current one-stage methods, which follow a DETR-style pipeline, achieve state-of-the-art (SOTA) performance with high-resolution inputs, we observe that this particularly benefits the estimation of individuals in smaller scales of the image (e.g., those far from the camera), but at the cost of significantly increased computation overhead. To address this, we introduce scale-adaptive tokens that are dynamically adjusted based on the relative scale of each individual in the image within the DETR framework. Specifically, individuals in smaller scales are processed at higher resolutions, larger ones at lower resolutions, and background regions are further distilled. These scale-adaptive tokens more efficiently encode the image features, facilitating subsequent decoding to regress the human mesh, while allowing the model to allocate computational resources more effectively and focus on more challenging cases. Experiments show that our method preserves the accuracy benefits of high-resolution processing while substantially reducing computational cost, achieving real-time inference with performance comparable to SOTA methods.