mlsanityData to 7 Oct 2026

HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning

arXiv 2603.02785 ↗First posted 3 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have been widely adopted in vision tasks due to their strong transferability. In Federated Learning (FL), where full fine-tuning is communication heavy, Low-Rank Adaptation (LoRA) provides an efficient and communication-friendly way to adapt ViTs. However, existing LoRA-based federated tuning methods overlook latent client structures in real-world settings, limiting shared representation learning and hindering effective adaptation to unseen clients. To address this, we propose HiLoRA, a hierarchical LoRA framework that places adapters at three levels: root, cluster, and leaf, each designed to capture global, subgroup, and client-specific knowledge, respectively. Through cross-tier orthogonality and cascaded optimization, HiLoRA separates update subspaces and aligns each tier with its residual personalized objective. In particular, we develop a LoRA-Subspace Adaptive Clustering mechanism that infers latent client groups via subspace similarity analysis, thereby facilitating knowledge sharing across structurally aligned clients. Theoretically, we establish a tier-wise generalization analysis that supports HiLoRA's design. Experiments on ViT backbones with CIFAR-100 and DomainNet demonstrate consistent improvements in both personalization and generalization.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

36 of 49 references.

References · 36

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Segment Anythingcited by 8,158
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2020A Survey on Visual Transformercited by 653
  8. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  9. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  10. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  11. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  12. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
24 more
  1. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 144
  2. 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
  3. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  4. 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
  5. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 59
  6. 2024FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuningcited by 57
  7. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  8. 2024Dual-Personalizing Adapter for Federated Foundation Modelscited by 54
  9. 2020FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measurecited by 45
  10. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
  11. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  12. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  13. 2024An Aggregation-Free Federated Learning for Tackling Data Heterogeneitycited by 33
  14. 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
  15. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
  16. 2023FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learningcited by 21
  17. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  18. 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
  19. 2023PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmarkcited by 16
  20. 2025FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Modelscited by 15
  21. 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRAcited by 13
  22. 2023ProtoFL: Unsupervised Federated Learning via Prototypical Distillationcited by 11
  23. 2025Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRAcited by 6
  24. 2025FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregationcited by 3

Built on by

Not cited yet.