Operator-Theoretic Framework for Gradient-Free Federated Learning
Not on the map. Why?
Abstract (arXiv)
Federated learning must address heterogeneity, strict communication and computation limits, and privacy while ensuring performance. We propose an operator-theoretic framework that maps the L2-optimal solution into a reproducing kernel Hilbert space (RKHS) via a forward operator, approximates it using available data, and maps back with the inverse operator, yielding a gradient-free scheme. Finite-sample bounds are derived using concentration inequalities over operator norms, and the framework identifies a data-dependent hypothesis space with guarantees on risk, error, robustness, and approximation. Within this space we design efficient kernel machines leveraging the space folding property of Kernel Affine Hull Machines. Clients transfer knowledge via a scalar space folding measure, reducing communication and enabling a simple differentially private protocol: summaries are computed from noise-perturbed data matrices in one step, avoiding per-round clipping and privacy accounting. The induced global rule requires only integer minimum and equality-comparison operations per test point, making it compatible with fully homomorphic encryption (FHE). Across four benchmarks, the gradient-free FL method with fixed encoder embeddings matches or outperforms strong gradient-based fine-tuning, with gains up to 23.7 points. In differentially private experiments, kernel smoothing mitigates accuracy loss in high-privacy regimes. The global rule admits an FHE realization using Q × C encrypted minimum and C equality-comparison operations per test point, with operation-level benchmarks showing practical latencies. Overall, the framework provides provable guarantees with low communication, supports private knowledge transfer via scalar summaries, and yields an FHE-compatible prediction rule offering a mathematically grounded alternative to gradient-based federated learning under heterogeneity.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 37 of its 74 references that are in the map. First posted on arXiv 30 Nov 2025.
Builds on
37 of 74 references.
References · 37
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
- 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
- 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
- 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
25 more
- 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
- 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
- 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
- 2020Understanding Gradient Clipping in Private SGD: A Geometric Perspectivecited by 140
- 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
- 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
- 2022Automatic Clipping: Differentially Private Deep Learning Made Easier and Strongercited by 87
- 2021Personalized Federated Learning with Gaussian Processescited by 76
- 2020Local and Central Differential Privacy for Robustness and Privacy in Federated Learningcited by 72
- 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
- 2020Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Frameworkcited by 59
- 2022Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streamscited by 59
- 2021Iterative Methods for Private Synthetic Data: Unifying Framework and New Methodscited by 58
- 2021Natural Language Understanding with Privacy-Preserving BERTcited by 54
- 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
- 2022Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learningcited by 48
- 2021FedSoft: Soft Clustered Federated Learning with Proximal Local Updatingcited by 46
- 2023FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning Systemcited by 38
- 2022Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspacescited by 36
- 2022FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reductioncited by 35
- 2023Personalized Online Federated Learning with Multiple Kernelscited by 11
- 2023FedDWA: Personalized Federated Learning with Dynamic Weight Adjustmentcited by 9
- 2020Revisiting Model-Agnostic Private Learning: Faster Rates and Active Learningcited by 4
- 2023On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approachcited by 3
- 2024Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descentcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.