# Self Supervised Networks for Learning Latent Space Representations of Human Body Scans and Motions

arXiv: https://arxiv.org/abs/2411.03475 · HTML: https://mlsanity.com/paper/2411.03475/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields](https://mlsanity.com/paper/2207.13807/) (2022)
- [LIMP: Learning Latent Shape Representations with Metric Preservation Priors](https://mlsanity.com/paper/2003.12283/) (2020)
- [Building Statistical Shape Spaces for 3D Human Modeling](https://mlsanity.com/paper/1503.05860/) (2015)
- [ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators](https://mlsanity.com/paper/2108.09432/) (2021)
- [The varifold representation of non-oriented shapes for diffeomorphic registration](https://mlsanity.com/paper/1304.6108/) (2013)
- [Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields](https://mlsanity.com/paper/2108.08931/) (2021)
- [Elastic shape analysis of surfaces with second-order Sobolev metrics: a comprehensive numerical framework](https://mlsanity.com/paper/2204.04238/) (2022)
- [GLASS: Geometric Latent Augmentation for Shape Spaces](https://mlsanity.com/paper/2108.03225/) (2021)
- [ResNet-LDDMM: Advancing the LDDMM Framework using Deep Residual Networks](https://mlsanity.com/paper/2102.07951/) (2021)
- [On Linear Interpolation in the Latent Space of Deep Generative Models](https://mlsanity.com/paper/2105.03663/) (2021)
- [A Riemannian Framework for Analysis of Human Body Surface](https://mlsanity.com/paper/2108.11449/) (2021)
- [3D Shape Sequence of Human Comparison and Classification using Current and Varifolds](https://mlsanity.com/paper/2207.12485/) (2022)
- [Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework](https://mlsanity.com/paper/2110.06421/) (2021)
- [BaRe-ESA: A Riemannian Framework for Unregistered Human Body Shapes](https://mlsanity.com/paper/2211.13185/) (2022)
- [Feature-Based Interpolation and Geodesics in the Latent Spaces of Generative Models](https://mlsanity.com/paper/1904.03445/) (2019)
- [Basis restricted elastic shape analysis on the space of unregistered surfaces](https://mlsanity.com/paper/2311.04382/) (2023)
- [VariGrad: A Novel Feature Vector Architecture for Geometric Deep Learning on Unregistered Data](https://mlsanity.com/paper/2307.03553/) (2023)
- [BLiSS: Bootstrapped Linear Shape Space](https://mlsanity.com/paper/2309.01765/) (2023)

## Built on by (most cited first, 3 of 3)

- [LiLa-Net: Lightweight Latent LiDAR Autoencoder for 3D Point Cloud Reconstruction](https://mlsanity.com/paper/2510.02028/) (2025)
- [ARAPDiffusion: ARAP Regularization for Diffusion-Based Deformable Shape Space Learning](https://mlsanity.com/paper/2606.06887/) (2026)
- [Learning the Geometry of Data: A Mathematical Review of Shape Space Analysis](https://mlsanity.com/paper/2606.17022/) (2026)

## Abstract (arXiv)

This paper introduces self-supervised neural network models to tackle several fundamental problems in the field of 3D human body analysis and processing. First, we propose VariShaPE (Varifold Shape Parameter Estimator), a novel architecture for the retrieval of latent space representations of body shapes and poses. This network offers a fast and robust method to estimate the embedding of arbitrary unregistered meshes into the latent space. Second, we complement the estimation of latent codes with MoGeN (Motion Geometry Network) a framework that learns the geometry on the latent space itself. This is achieved by lifting the body pose parameter space into a higher dimensional Euclidean space in which body motion mini-sequences from a training set of 4D data can be approximated by simple linear interpolation. Using the SMPL latent space representation we illustrate how the combination of these network models, once trained, can be used to perform a variety of tasks with very limited computational cost. This includes operations such as motion interpolation, extrapolation and transfer as well as random shape and pose generation.