# Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing

arXiv: https://arxiv.org/abs/2309.04557 · HTML: https://mlsanity.com/paper/2309.04557/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 70 references that are in the map.
- First posted on arXiv 8 Sep 2023.

## Builds on (most cited first, 30 of 36)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Tighter Theory for Local SGD on Identical and Heterogeneous Data](https://mlsanity.com/paper/1909.04746/) (2019)
- [Linearized two-layers neural networks in high dimension](https://mlsanity.com/paper/1904.12191/) (2019)
- [Stochastic modified equations and adaptive stochastic gradient algorithms](https://mlsanity.com/paper/1511.06251/) (2015)
- [Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration](https://mlsanity.com/paper/2101.10588/) (2021)
- [PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees](https://mlsanity.com/paper/2002.05551/) (2020)
- [Maximum Principle Based Algorithms for Deep Learning](https://mlsanity.com/paper/1710.09513/) (2017)
- [Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/1811.02050/) (2018)
- [Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery](https://mlsanity.com/paper/1711.09082/) (2017)
- [Approximation Bounds for Random Neural Networks and Reservoir Systems](https://mlsanity.com/paper/2002.05933/) (2020)
- [FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy](https://mlsanity.com/paper/2302.10429/) (2023)
- [Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality](https://mlsanity.com/paper/2106.08900/) (2021)
- [Optimal Stopping via Randomized Neural Networks](https://mlsanity.com/paper/2104.13669/) (2021)
- [Discrete-time signatures and randomness in reservoir computing](https://mlsanity.com/paper/2010.14615/) (2020)
- [Solving high-dimensional optimal stopping problems using deep learning](https://mlsanity.com/paper/1908.01602/) (2019)
- [Risk bounds for reservoir computing](https://mlsanity.com/paper/1910.13886/) (2019)
- [A Wasserstein distance approach for concentration of empirical risk estimates](https://mlsanity.com/paper/1902.10709/) (2019)
- [Tighter Regret Analysis and Optimization of Online Federated Learning](https://mlsanity.com/paper/2205.06491/) (2022)
- [Powers of Tensors and Fast Matrix Multiplication](https://mlsanity.com/paper/1401.7714/) (2014)
- [Deep Learning for Ranking Response Surfaces with Applications to Optimal Stopping Problems](https://mlsanity.com/paper/1901.03478/) (2019)
- [MARS: Meta-Learning as Score Matching in the Function Space](https://mlsanity.com/paper/2210.13319/) (2022)
- [Spectrally-truncated kernel ridge regression and its free lunch](https://mlsanity.com/paper/1906.06276/) (2019)
- [Linear quadratic mean field social optimization: Asymptotic solvability and decentralized control](https://mlsanity.com/paper/2012.15468/) (2020)

## Built on by (most cited first, 2 of 2)

- [Universal approximation property of Banach space-valued random feature models including random neural networks](https://mlsanity.com/paper/2312.08410/) (2023)
- [Solving stochastic partial differential equations using neural networks in the Wiener chaos expansion](https://mlsanity.com/paper/2411.03384/) (2024)

## Abstract (arXiv)

We propose an optimal iterative scheme for federated transfer learning, where a central planner has access to datasets cal D1,dots,cal DN for the same learning model fθ. Our objective is to minimize the cumulative deviation of the generated parameters θi(t)t=0T across all T iterations from the specialized parameters θ^star1,ldots,θ^starN obtained for each dataset, while respecting the loss function for the model fθ(T) produced by the algorithm upon halting. We only allow for continual communication between each of the specialized models (nodes/agents) and the central planner (server), at each iteration (round). For the case where the model fθ is a finite-rank kernel regression, we derive explicit updates for the regret-optimal algorithm. By leveraging symmetries within the regret-optimal algorithm, we further develop a nearly regret-optimal heuristic that runs with O(Np2) fewer elementary operations, where p is the dimension of the parameter space. Additionally, we investigate the adversarial robustness of the regret-optimal algorithm showing that an adversary which perturbs q training pairs by at-most varepsilon>0, across all training sets, cannot reduce the regret-optimal algorithm's regret by more than O(varepsilon q barN1/2), where barN is the aggregate number of training pairs. To validate our theoretical findings, we conduct numerical experiments in the context of American option pricing, utilizing a randomly generated finite-rank kernel.