# Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descent

arXiv: https://arxiv.org/abs/2407.04335 · HTML: https://mlsanity.com/paper/2407.04335/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 5 Jul 2024.

## Builds on (most cited first, 30 of 30)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Tighter Theory for Local SGD on Identical and Heterogeneous Data](https://mlsanity.com/paper/1909.04746/) (2019)
- [To understand deep learning we need to understand kernel learning](https://mlsanity.com/paper/1802.01396/) (2018)
- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [FALKON: An Optimal Large Scale Kernel Method](https://mlsanity.com/paper/1705.10958/) (2017)
- [Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces](https://mlsanity.com/paper/2209.10526/) (2022)
- [Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks](https://mlsanity.com/paper/2209.15595/) (2022)
- [Efficient Learning of Domain-invariant Image Representations](https://mlsanity.com/paper/1301.3224/) (2013)
- [Learning an Invariant Hilbert Space for Domain Adaptation](https://mlsanity.com/paper/1611.08350/) (2016)
- [Optimal Bayesian Transfer Learning](https://mlsanity.com/paper/1801.00857/) (2018)
- [Personalized Online Federated Learning with Multiple Kernels](https://mlsanity.com/paper/2311.05108/) (2023)
- [Autoencoding any Data through Kernel Autoencoders](https://mlsanity.com/paper/1805.11028/) (2018)
- [The Freiburg Groceries Dataset](https://mlsanity.com/paper/1611.05799/) (2016)
- [Membership-Mappings for Data Representation Learning: Measure Theoretic Conceptualization](https://mlsanity.com/paper/2104.07060/) (2021)
- [Differentially Private Transferrable Deep Learning with Membership-Mappings](https://mlsanity.com/paper/2105.04615/) (2021)
- [A Unified Linear Speedup Analysis of Federated Averaging and Nesterov FedAvg](https://mlsanity.com/paper/2007.05690/) (2020)
- [The Deep Kernelized Autoencoder](https://mlsanity.com/paper/1807.07868/) (2018)
- [On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approach](https://mlsanity.com/paper/2304.01300/) (2023)
- [Membership-Mappings for Practical Secure Distributed Deep Learning](https://mlsanity.com/paper/2204.05765/) (2022)

## Built on by (most cited first, 2 of 2)

- [Operator-Theoretic Framework for Gradient-Free Federated Learning](https://mlsanity.com/paper/2512.01025/) (2025)
- [Kernel Affine Hull Machines as Compute-Efficient Encoders for Frozen Semantic Spaces](https://mlsanity.com/paper/2605.02950/) (2026)

## Abstract (arXiv)

This paper develops a novel mathematical framework for collaborative learning by means of geometrically inspired kernel machines which includes statements on the bounds of generalisation and approximation errors, and sample complexity. For classification problems, this approach allows us to learn bounded geometric structures around given data points and hence solve the global model learning problem in an efficient way by exploiting convexity properties of the related optimisation problem in a Reproducing Kernel Hilbert Space (RKHS). In this way, we can reduce classification problems to determining the closest bounded geometric structure from a given data point. Further advantages that come with our solution is that our approach does not require clients to perform multiple epochs of local optimisation using stochastic gradient descent, nor require rounds of communication between client/server for optimising the global model. We highlight that numerous experiments have shown that the proposed method is a competitive alternative to the state-of-the-art.