# Representation learning of vertex heatmaps for 3D human mesh reconstruction from multi-view images

arXiv: https://arxiv.org/abs/2306.16615 · HTML: https://mlsanity.com/paper/2306.16615/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes](https://mlsanity.com/paper/2104.07300/) (2021)
- [Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation](https://mlsanity.com/paper/2004.00329/) (2020)
- [Learnable human mesh triangulation for 3D human pose and shape estimation](https://mlsanity.com/paper/2208.11251/) (2022)
- [Multi-view Human Pose and Shape Estimation Using Learnable Volumetric Aggregation](https://mlsanity.com/paper/2011.13427/) (2020)

## Built on by (most cited first, 4 of 4)

- [Sparse2Dense: A Keypoint-driven Generative Framework for Human Video Compression and Vertex Prediction](https://mlsanity.com/paper/2509.23169/) (2025)
- [Two Views Are Better than One: Monocular 3D Pose Estimation with Multiview Consistency](https://mlsanity.com/paper/2311.12421/) (2023)
- [Bidirectional Regression for Monocular 6DoF Head Pose Estimation and Reference System Alignment](https://mlsanity.com/paper/2407.14136/) (2024)
- [PCFEx: Point Cloud Feature Extraction for Graph Neural Networks](https://mlsanity.com/paper/2603.08540/) (2026)

## Abstract (arXiv)

This study addresses the problem of 3D human mesh reconstruction from multi-view images. Recently, approaches that directly estimate the skinned multi-person linear model (SMPL)-based human mesh vertices based on volumetric heatmap representation from input images have shown good performance. We show that representation learning of vertex heatmaps using an autoencoder helps improve the performance of such approaches. Vertex heatmap autoencoder (VHA) learns the manifold of plausible human meshes in the form of latent codes using AMASS, which is a large-scale motion capture dataset. Body code predictor (BCP) utilizes the learned body prior from VHA for human mesh reconstruction from multi-view images through latent code-based supervision and transfer of pretrained weights. According to experiments on Human3.6M and LightStage datasets, the proposed method outperforms previous methods and achieves state-of-the-art human mesh reconstruction performance.