# Neural Jacobian Fields: Learning Intrinsic Mappings of Arbitrary Meshes

arXiv: https://arxiv.org/abs/2205.02904 · HTML: https://mlsanity.com/paper/2205.02904/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 82 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [SDM-NET: Deep Generative Network for Structured Deformable Mesh](https://mlsanity.com/paper/1908.04520/) (2019)
- [Deep Geometric Prior for Surface Reconstruction](https://mlsanity.com/paper/1811.10943/) (2018)
- [Neural Cages for Detail-Preserving 3D Deformations](https://mlsanity.com/paper/1912.06395/) (2019)
- [Variational Autoencoders for Deforming 3D Mesh Models](https://mlsanity.com/paper/1709.04307/) (2017)
- [3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations](https://mlsanity.com/paper/2108.12958/) (2021)
- [Learning Skeletal Articulations with Neural Blend Shapes](https://mlsanity.com/paper/2105.02451/) (2021)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [Deformable Shape Completion with Graph Convolutional Autoencoders](https://mlsanity.com/paper/1712.00268/) (2017)
- [ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators](https://mlsanity.com/paper/2108.09432/) (2021)
- [Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets](https://mlsanity.com/paper/1908.08506/) (2019)
- [Shape Reconstruction by Learning Differentiable Surface Representations](https://mlsanity.com/paper/1911.11227/) (2019)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [Neural Surface Maps](https://mlsanity.com/paper/2103.16942/) (2021)
- [RigNet: Neural Rigging for Articulated Characters](https://mlsanity.com/paper/2005.00559/) (2020)
- [High-order Differentiable Autoencoder for Nonlinear Model Reduction](https://mlsanity.com/paper/2102.11026/) (2021)

## Built on by (most cited first, 28 of 28)

- [TextDeformer: Geometry Manipulation using Text Guidance](https://mlsanity.com/paper/2304.13348/) (2023)
- [Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features](https://mlsanity.com/paper/2311.17024/) (2023)
- [Garment3DGen: 3D Garment Stylization and Texture Generation](https://mlsanity.com/paper/2403.18816/) (2024)
- [Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild](https://mlsanity.com/paper/2305.08296/) (2023)
- [As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priors](https://mlsanity.com/paper/2311.16739/) (2023)
- [MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation](https://mlsanity.com/paper/2408.14899/) (2024)
- [HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation](https://mlsanity.com/paper/2403.09326/) (2024)
- [Advances in 3D Neural Stylization: A Survey](https://mlsanity.com/paper/2311.18328/) (2023)
- [Neural Garment Dynamics via Manifold-Aware Transformers](https://mlsanity.com/paper/2407.06101/) (2024)
- [Variational Barycentric Coordinates](https://mlsanity.com/paper/2310.03861/) (2023)
- [Explorable Mesh Deformation Subspaces from Unstructured Generative Models](https://mlsanity.com/paper/2310.07814/) (2023)
- [SMF: Template-free and Rig-free Animation Transfer using Kinetic Codes](https://mlsanity.com/paper/2504.04831/) (2025)
- [PoissonNet: A Local-Global Approach for Learning on Surfaces](https://mlsanity.com/paper/2510.14146/) (2025)
- [Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads](https://mlsanity.com/paper/2410.11041/) (2024)
- [Surface-Aware Distilled 3D Semantic Features](https://mlsanity.com/paper/2503.18254/) (2025)
- [Neural Face Skinning for Mesh-agnostic Facial Expression Cloning](https://mlsanity.com/paper/2505.22416/) (2025)
- [Non-Rigid 3D Shape Correspondences: From Foundations to Open Challenges and Opportunities](https://mlsanity.com/paper/2604.01274/) (2026)
- [Sketch-guided Cage-based 3D Gaussian Splatting Deformation](https://mlsanity.com/paper/2411.12168/) (2024)
- [Controllable Shape Modeling with Neural Generalized Cylinder](https://mlsanity.com/paper/2410.03675/) (2024)
- [CRAFT: Designing Creative and Functional 3D Objects](https://mlsanity.com/paper/2412.03889/) (2024)
- [ScanMove: Motion Prediction and Transfer for Unregistered Body Meshes](https://mlsanity.com/paper/2508.21095/) (2025)
- [MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation](https://mlsanity.com/paper/2405.20155/) (2024)
- [Neural Kinematic Bases for Fluids](https://mlsanity.com/paper/2504.15657/) (2025)
- [Patch-based Representation and Learning for Efficient Deformation Modeling](https://mlsanity.com/paper/2601.05035/) (2026)
- [Learning Domain Agnostic Latent Embeddings of 3D Faces for Zero-shot Animal Expression Transfer](https://mlsanity.com/paper/2601.06484/) (2026)
- [Mat'ern Noise for Triangulation-Agnostic Flow Matching on Meshes](https://mlsanity.com/paper/2605.19305/) (2026)
- [PoseAlign: Sculpting Pose-Consistent Meshes via Text-Guided Deformation](https://mlsanity.com/paper/2607.10560/) (2026)
- [As-Rigid-As-Possible Regularization for Implicit Surfaces](https://mlsanity.com/paper/2608.15933/) (2026)

## Abstract (arXiv)

This paper introduces a framework designed to accurately predict piecewise linear mappings of arbitrary meshes via a neural network, enabling training and evaluating over heterogeneous collections of meshes that do not share a triangulation, as well as producing highly detail-preserving maps whose accuracy exceeds current state of the art. The framework is based on reducing the neural aspect to a prediction of a matrix for a single given point, conditioned on a global shape descriptor. The field of matrices is then projected onto the tangent bundle of the given mesh, and used as candidate jacobians for the predicted map. The map is computed by a standard Poisson solve, implemented as a differentiable layer with cached pre-factorization for efficient training. This construction is agnostic to the triangulation of the input, thereby enabling applications on datasets with varying triangulations. At the same time, by operating in the intrinsic gradient domain of each individual mesh, it allows the framework to predict highly-accurate mappings. We validate these properties by conducting experiments over a broad range of scenarios, from semantic ones such as morphing, registration, and deformation transfer, to optimization-based ones, such as emulating elastic deformations and contact correction, as well as being the first work, to our knowledge, to tackle the task of learning to compute UV parameterizations of arbitrary meshes. The results exhibit the high accuracy of the method as well as its versatility, as it is readily applied to the above scenarios without any changes to the framework.