# Deep Virtual Markers for Articulated 3D Shapes

arXiv: https://arxiv.org/abs/2108.09000 · HTML: https://mlsanity.com/paper/2108.09000/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 77 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 30 of 35)

- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels](https://mlsanity.com/paper/1711.08920/) (2017)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Deep Functional Maps: Structured Prediction for Dense Shape Correspondence](https://mlsanity.com/paper/1704.08686/) (2017)
- [Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence](https://mlsanity.com/paper/2003.14286/) (2020)
- [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020)
- [Efficient Learning on Point Clouds with Basis Point Sets](https://mlsanity.com/paper/1908.09186/) (2019)
- [Learning elementary structures for 3D shape generation and matching](https://mlsanity.com/paper/1908.04725/) (2019)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor](https://mlsanity.com/paper/1804.06023/) (2018)
- [LiveCap: Real-time Human Performance Capture from Monocular Video](https://mlsanity.com/paper/1810.02648/) (2018)
- [SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator](https://mlsanity.com/paper/1911.05856/) (2019)
- [Unsupervised Deep Learning for Structured Shape Matching](https://mlsanity.com/paper/1812.03794/) (2018)
- [Continuous and Orientation-preserving Correspondences via Functional Maps](https://mlsanity.com/paper/1806.04455/) (2018)
- [Deep Shells: Unsupervised Shape Correspondence with Optimal Transport](https://mlsanity.com/paper/2010.15261/) (2020)
- [Smooth Shells: Multi-Scale Shape Registration with Functional Maps](https://mlsanity.com/paper/1905.12512/) (2019)
- [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019)
- [Correspondence Learning via Linearly-invariant Embedding](https://mlsanity.com/paper/2010.13136/) (2020)
- [Joint Multi-Person Pose Estimation and Semantic Part Segmentation](https://mlsanity.com/paper/1708.03383/) (2017)
- [A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes](https://mlsanity.com/paper/1809.06664/) (2018)
- [Cyclic Functional Mapping: Self-supervised correspondence between non-isometric deformable shapes](https://mlsanity.com/paper/1912.01249/) (2019)

## Built on by (most cited first, 7 of 7)

- [Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization](https://mlsanity.com/paper/2111.12878/) (2021)
- [NICP: Neural ICP for 3D Human Registration at Scale](https://mlsanity.com/paper/2312.14024/) (2023)
- [Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration](https://mlsanity.com/paper/2311.05828/) (2023)
- [Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learning](https://mlsanity.com/paper/2306.11406/) (2023)
- [Putting 3D Spatially Sparse Networks on a Diet](https://mlsanity.com/paper/2112.01316/) (2021)
- [Skin Lesion Correspondence Localization in Total Body Photography](https://mlsanity.com/paper/2307.09642/) (2023)
- [Revisiting Lesion Tracking in 3D Total Body Photography](https://mlsanity.com/paper/2412.07132/) (2024)

## Abstract (arXiv)

We propose deep virtual markers, a framework for estimating dense and accurate positional information for various types of 3D data. We design a concept and construct a framework that maps 3D points of 3D articulated models, like humans, into virtual marker labels. To realize the framework, we adopt a sparse convolutional neural network and classify 3D points of an articulated model into virtual marker labels. We propose to use soft labels for the classifier to learn rich and dense interclass relationships based on geodesic distance. To measure the localization accuracy of the virtual markers, we test FAUST challenge, and our result outperforms the state-of-the-art. We also observe outstanding performance on the generalizability test, unseen data evaluation, and different 3D data types (meshes and depth maps). We show additional applications using the estimated virtual markers, such as non-rigid registration, texture transfer, and realtime dense marker prediction from depth maps.