# VINECS: Video-based Neural Character Skinning

arXiv: https://arxiv.org/abs/2307.00842 · HTML: https://mlsanity.com/paper/2307.00842/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 69 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [TAVA: Template-free Animatable Volumetric Actors](https://mlsanity.com/paper/2206.08929/) (2022)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [ARAH: Animatable Volume Rendering of Articulated Human SDFs](https://mlsanity.com/paper/2210.10036/) (2022)
- [Real-time Deep Dynamic Characters](https://mlsanity.com/paper/2105.01794/) (2021)
- [Background Matting: The World is Your Green Screen](https://mlsanity.com/paper/2004.00626/) (2020)
- [LiveCap: Real-time Human Performance Capture from Monocular Video](https://mlsanity.com/paper/1810.02648/) (2018)
- [Learning Skeletal Articulations with Neural Blend Shapes](https://mlsanity.com/paper/2105.02451/) (2021)
- [Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture](https://mlsanity.com/paper/2011.12866/) (2020)
- [Skeleton-free Pose Transfer for Stylized 3D Characters](https://mlsanity.com/paper/2208.00790/) (2022)
- [SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning Prediction of Synthetic Characters](https://mlsanity.com/paper/2203.04746/) (2022)
- [HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances](https://mlsanity.com/paper/2210.05665/) (2022)
- [Object Wake-up: 3D Object Rigging from a Single Image](https://mlsanity.com/paper/2108.02708/) (2021)

## Built on by (most cited first, 6 of 6)

- [Anymate: A Dataset and Baselines for Learning 3D Object Rigging](https://mlsanity.com/paper/2505.06227/) (2025)
- [HRAvatar: High-Quality and Relightable Gaussian Head Avatar](https://mlsanity.com/paper/2503.08224/) (2025)
- [Robust Biharmonic Skinning Using Geometric Fields](https://mlsanity.com/paper/2406.00238/) (2024)
- [On the Skinning of Gaussian Avatars](https://mlsanity.com/paper/2509.11411/) (2025)
- [SOMA: From Surface Observations to Muscle Anatomy](https://mlsanity.com/paper/2606.09246/) (2026)
- [LatentReRig: An SDF-Based VAE with Dual Decoders for Latent-Space Deformation Conditioning](https://mlsanity.com/paper/2609.31720/) (2026)

## Abstract (arXiv)

Rigging and skinning clothed human avatars is a challenging task and traditionally requires a lot of manual work and expertise. Recent methods addressing it either generalize across different characters or focus on capturing the dynamics of a single character observed under different pose configurations. However, the former methods typically predict solely static skinning weights, which perform poorly for highly articulated poses, and the latter ones either require dense 3D character scans in different poses or cannot generate an explicit mesh with vertex correspondence over time. To address these challenges, we propose a fully automated approach for creating a fully rigged character with pose-dependent skinning weights, which can be solely learned from multi-view video. Therefore, we first acquire a rigged template, which is then statically skinned. Next, a coordinate-based MLP learns a skinning weights field parameterized over the position in a canonical pose space and the respective pose. Moreover, we introduce our pose- and view-dependent appearance field allowing us to differentiably render and supervise the posed mesh using multi-view imagery. We show that our approach outperforms state-of-the-art while not relying on dense 4D scans.