# Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces

arXiv: https://arxiv.org/abs/2501.18373 · HTML: https://mlsanity.com/paper/2501.18373/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Zero-Shot Reinforcement Learning via Function Encoders](https://mlsanity.com/paper/2401.17173/) (2024)
- [On the Computational Intractability of Exact and Approximate Dictionary Learning](https://mlsanity.com/paper/1405.6664/) (2014)
- [Understanding the Transferability of Representations via Task-Relatedness](https://mlsanity.com/paper/2307.00823/) (2023)
- [Zero-Shot Transfer of Neural ODEs](https://mlsanity.com/paper/2405.08954/) (2024)
- [Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks](https://mlsanity.com/paper/2310.06369/) (2023)
- [Transfer Learning with Kernel Methods](https://mlsanity.com/paper/2211.00227/) (2022)

## Built on by (most cited first, 10 of 10)

- [MoS-VLA: A Vision-Language-Action Model with One-Shot Skill Adaptation](https://mlsanity.com/paper/2510.16617/) (2025)
- [Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shifts](https://mlsanity.com/paper/2506.11033/) (2025)
- [Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems](https://mlsanity.com/paper/2509.18404/) (2025)
- [Function Spaces Without Kernels: Learning Compact Hilbert Space Representations](https://mlsanity.com/paper/2509.20605/) (2025)
- [Safety Generalization Under Distribution Shift in Safe Reinforcement Learning: A Diabetes Testbed](https://mlsanity.com/paper/2601.21094/) (2026)
- [Zero-Shot Function Encoder-Based Differentiable Predictive Control](https://mlsanity.com/paper/2511.05757/) (2025)
- [Learning Generalizable Neural Operators for Inverse Problems](https://mlsanity.com/paper/2512.18120/) (2025)
- [A Flow Matching Algorithm for Many-Shot Adaptation to Unseen Distributions](https://mlsanity.com/paper/2605.06272/) (2026)
- [OnlyDense: Reduced-Order Modeling for Lagrangian simulation](https://mlsanity.com/paper/2606.09065/) (2026)
- [Universal Sample-Based Embeddings of Higher-Order Objects](https://mlsanity.com/paper/2610.19899/) (2026)

## Abstract (arXiv)

A central challenge in transfer learning is designing algorithms that can quickly adapt and generalize to new tasks without retraining. Yet, the conditions of when and how algorithms can effectively transfer to new tasks is poorly characterized. We introduce a geometric characterization of transfer in Hilbert spaces and define three types of inductive transfer: interpolation within the convex hull, extrapolation to the linear span, and extrapolation outside the span. We propose a method grounded in the theory of function encoders to achieve all three types of transfer. Specifically, we introduce a novel training scheme for function encoders using least-squares optimization, prove a universal approximation theorem for function encoders, and provide a comprehensive comparison with existing approaches such as transformers and meta-learning on four diverse benchmarks. Our experiments demonstrate that the function encoder outperforms state-of-the-art methods on four benchmark tasks and on all three types of transfer.