# FedLoRD: Personalized Federated Learning based on Low-Rank Structures of Model Parameter Differences

arXiv: https://arxiv.org/abs/2610.17621 · HTML: https://mlsanity.com/paper/2610.17621/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Convolutional neural networks with low-rank regularization](https://mlsanity.com/paper/1511.06067/) (2015)
- [PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmark](https://mlsanity.com/paper/2312.04992/) (2023)
- [Sparse Personalized Federated Learning](https://mlsanity.com/paper/2107.05330/) (2021)
- [pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup](https://mlsanity.com/paper/2501.11002/) (2025)
- [FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks](https://mlsanity.com/paper/2501.16379/) (2025)

## Abstract (arXiv)

We propose FedLoRD, a personalized federated learning (PFL) method based on the low-rank structure of the differences between the global and personalized model parameters. PFL is a branch of federated learning that aims to learn models tailored to the local data of each client under biased local data distributions. In its local training, each client learns a personalized model by regularizing the difference between its model parameters and the global ones sent from the server. The regularization function needs to be designed with priors on the model parameter difference; however, the existing PFL methods rely on empirical choices (e.g., ℓ1- and ℓ2-norms), which do not take into account the actual characteristics of the difference. The resulting mismatch of priors can lead to suboptimal personalized models. To address this issue, we propose a PFL method whose regularizer is built on an observed characteristic of the model parameter differences: they are approximately low-rank at every layer. In our method, we first formulate a local training problem with nuclear norm regularization on the layer-wise differences. We further rewrite this problem via the factorized representation of the nuclear norm so that it can be efficiently solved by a plain stochastic gradient descent method. Image classification experiments on three benchmark datasets demonstrate that FedLoRD outperforms several existing PFL methods.