# Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks

arXiv: https://arxiv.org/abs/2310.06369 · HTML: https://mlsanity.com/paper/2310.06369/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Learning Multimodal Graph-to-Graph Translation for Molecular Optimization](https://mlsanity.com/paper/1812.01070/) (2018)
- [On Representation Knowledge Distillation for Graph Neural Networks](https://mlsanity.com/paper/2111.04964/) (2021)
- [Structured Multi-task Learning for Molecular Property Prediction](https://mlsanity.com/paper/2203.04695/) (2022)
- [Unsupervised Domain Adaptation via Discriminative Manifold Embedding and Alignment](https://mlsanity.com/paper/2002.08675/) (2020)
- [Grouping-matrix based Graph Pooling with Adaptive Number of Clusters](https://mlsanity.com/paper/2209.02939/) (2022)
- [MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction](https://mlsanity.com/paper/2305.01912/) (2023)
- [Coordinating Cross-modal Distillation for Molecular Property Prediction](https://mlsanity.com/paper/2211.16712/) (2022)

## Built on by (most cited first, 6 of 6)

- [Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces](https://mlsanity.com/paper/2501.18373/) (2025)
- [Multitask Extension of Geometrically Aligned Transfer Encoder](https://mlsanity.com/paper/2405.01974/) (2024)
- [Task Addition in Multi-Task Learning by Geometrical Alignment](https://mlsanity.com/paper/2409.16645/) (2024)
- [Scalable Multi-Task Transfer Learning for Molecular Property Prediction](https://mlsanity.com/paper/2410.00432/) (2024)
- [Geometric Embedding Alignment via Curvature Matching in Transfer Learning](https://mlsanity.com/paper/2506.13015/) (2025)
- [Towards a Generalizable AI for Materials Discovery: Validation through Immersion Coolant Screening](https://mlsanity.com/paper/2510.23371/) (2025)

## Abstract (arXiv)

Transfer learning is a crucial technique for handling a small amount of data that is potentially related to other abundant data. However, most of the existing methods are focused on classification tasks using images and language datasets. Therefore, in order to expand the transfer learning scheme to regression tasks, we propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer Encoder (GATE). In this method, we interpret the latent vectors from the model to exist on a Riemannian curved manifold. We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data. This also serves as an effective regularizer for the model to behave in extrapolation regions. In this article, we demonstrate that GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets.