# Transfer Learning using Spectral Convolutional Autoencoders on Semi-Regular Surface Meshes

arXiv: https://arxiv.org/abs/2212.05810 · HTML: https://mlsanity.com/paper/2212.05810/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation](https://mlsanity.com/paper/1905.02876/) (2019)
- [Subdivision-Based Mesh Convolution Networks](https://mlsanity.com/paper/2106.02285/) (2021)
- [Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels](https://mlsanity.com/paper/2006.04325/) (2020)
- [Deformable Shape Completion with Graph Convolutional Autoencoders](https://mlsanity.com/paper/1712.00268/) (2017)
- [Neural Subdivision](https://mlsanity.com/paper/2005.01819/) (2020)
- [Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes](https://mlsanity.com/paper/2110.09401/) (2021)
- [Analysis and Prediction of Deforming 3D Shapes using Oriented Bounding Boxes and LSTM Autoencoders](https://mlsanity.com/paper/2009.03782/) (2020)

## Built on by (most cited first, 3 of 3)

- [WrappingNet: Mesh Autoencoder via Deep Sphere Deformation](https://mlsanity.com/paper/2308.15413/) (2023)
- [Unsupervised Representation Learning for Diverse Deformable Shape Collections](https://mlsanity.com/paper/2310.18141/) (2023)
- [A 3D mesh convolution-based autoencoder for geometry compression](https://mlsanity.com/paper/2603.02125/) (2026)

## Abstract (arXiv)

The underlying dynamics and patterns of 3D surface meshes deforming over time can be discovered by unsupervised learning, especially autoencoders, which calculate low-dimensional embeddings of the surfaces. To study the deformation patterns of unseen shapes by transfer learning, we want to train an autoencoder that can analyze new surface meshes without training a new network. Here, most state-of-the-art autoencoders cannot handle meshes of different connectivity and therefore have limited to no generalization capacities to new meshes. Also, reconstruction errors strongly increase in comparison to the errors for the training shapes. To address this, we propose a novel spectral CoSMA (Convolutional Semi-Regular Mesh Autoencoder) network. This patch-based approach is combined with a surface-aware training. It reconstructs surfaces not presented during training and generalizes the deformation behavior of the surfaces' patches. The novel approach reconstructs unseen meshes from different datasets in superior quality compared to state-of-the-art autoencoders that have been trained on these shapes. Our transfer learning errors on unseen shapes are 40% lower than those from models learned directly on the data. Furthermore, baseline autoencoders detect deformation patterns of unseen mesh sequences only for the whole shape. In contrast, due to the employed regional patches and stable reconstruction quality, we can localize where on the surfaces these deformation patterns manifest.