# NeuralAnnot: Neural Annotator for 3D Human Mesh Training Sets

arXiv: https://arxiv.org/abs/2011.11232 · HTML: https://mlsanity.com/paper/2011.11232/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2020.

## Built on by (most cited first, 30 of 62)

- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion](https://mlsanity.com/paper/2312.07531/) (2023)
- [SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2309.17448/) (2023)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer](https://mlsanity.com/paper/2303.16160/) (2023)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot](https://mlsanity.com/paper/2402.14654/) (2024)
- [TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation](https://mlsanity.com/paper/2404.16752/) (2024)
- [SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling](https://mlsanity.com/paper/2303.17368/) (2023)
- [AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2403.17934/) (2024)
- [Bringing Inputs to Shared Domains for 3D Interacting Hands Recovery in the Wild](https://mlsanity.com/paper/2303.13652/) (2023)
- [A Dataset of Relighted 3D Interacting Hands](https://mlsanity.com/paper/2310.17768/) (2023)
- [SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2501.09782/) (2025)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Video](https://mlsanity.com/paper/2308.10305/) (2023)
- [3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera](https://mlsanity.com/paper/2312.14157/) (2023)
- [DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery](https://mlsanity.com/paper/2404.01424/) (2024)
- [SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens](https://mlsanity.com/paper/2411.19824/) (2024)
- [EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activity](https://mlsanity.com/paper/2502.18373/) (2025)
- [VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space](https://mlsanity.com/paper/2312.08291/) (2023)
- [Authentic Hand Avatar from a Phone Scan via Universal Hand Model](https://mlsanity.com/paper/2405.07933/) (2024)
- [JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery](https://mlsanity.com/paper/2307.16377/) (2023)
- [MPT: Mesh Pre-Training with Transformers for Human Pose and Mesh Reconstruction](https://mlsanity.com/paper/2211.13357/) (2022)
- [Human Mesh Modeling for Anny Body](https://mlsanity.com/paper/2511.03589/) (2025)
- [Three Recipes for Better 3D Pseudo-GTs of 3D Human Mesh Estimation in the Wild](https://mlsanity.com/paper/2304.04875/) (2023)
- [ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning](https://mlsanity.com/paper/2307.01200/) (2023)
- [STAF: 3D Human Mesh Recovery from Video with Spatio-Temporal Alignment Fusion](https://mlsanity.com/paper/2401.01730/) (2024)
- [Body Knowledge and Uncertainty Modeling for Monocular 3D Human Body Reconstruction](https://mlsanity.com/paper/2308.00799/) (2023)
- [MeshPose: Unifying DensePose and 3D Body Mesh reconstruction](https://mlsanity.com/paper/2406.10180/) (2024)

## Abstract (arXiv)

Most 3D human mesh regressors are fully supervised with 3D pseudo-GT human model parameters and weakly supervised with GT 2D/3D joint coordinates as the 3D pseudo-GTs bring great performance gain. The 3D pseudo-GTs are obtained by annotators, systems that iteratively fit 3D human model parameters to GT 2D/3D joint coordinates of training sets in the pre-processing stage of the regressors. The fitted 3D parameters at the last fitting iteration become the 3D pseudo-GTs, used to fully supervise the regressors. Optimization-based annotators, such as SMPLify-X, have been widely used to obtain the 3D pseudo-GTs. However, they often produce wrong 3D pseudo-GTs as they fit the 3D parameters to GT of each sample independently. To overcome the limitation, we present NeuralAnnot, a neural network-based annotator. The main idea of NeuralAnnot is to employ a neural network-based regressor and dedicate it for the annotation. Assuming no 3D pseudo-GTs are available, NeuralAnnot is weakly supervised with GT 2D/3D joint coordinates of training sets. The testing results on the same training sets become 3D pseudo-GTs, used to fully supervise the regressors. We show that 3D pseudo-GTs of NeuralAnnot are highly beneficial to train the regressors. We made our 3D pseudo-GTs publicly available.