# Co-LoRA: Collaborative Model Personalization on Heterogeneous Multi-Modal Clients

arXiv: https://arxiv.org/abs/2506.11024 · HTML: https://mlsanity.com/paper/2506.11024/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agentic AI for wireless networks, one of 16 directions in LLM agents and tool use.
- First posted on arXiv 20 May 2025.

## Placed in

- LLM agents and tool use / [Agentic AI for wireless networks](https://mlsanity.com/agents/agentic-ai-for-wireless-networks/)

## Built on by (most cited first, 1 of 1)

- [FedSDR: Federated Self-Distillation with Rectification](https://mlsanity.com/paper/2605.18028/) (2026)

## Abstract (arXiv)

As AI becomes more personal, e.g., Agentic AI, there is an increasing need for personalizing models for various use cases. Personalized federated learning (PFL) enables each client to collaboratively leverage other clients' knowledge for better adaptation to the task of interest, without privacy risks. Despite its potential, existing PFL methods remain confined to rather simplified scenarios where data and models are the same across clients. To move towards realistic scenarios, we move beyond these restrictive assumptions by addressing both data and model heterogeneity. We propose a task-relevance-aware model aggregation strategy to reduce parameter interference under heterogeneous data. Moreover, we introduce Co-LoRA, a dimension-invariant module that enables knowledge sharing across heterogeneous architectures. To mimic the real-world task diversity, we propose a multi-modal PFL benchmark spanning 40 distinct tasks with distribution shifts over time. Extensive experiments shows that our proposed method significantly outperforms the state-of-the-art PFL methods under heterogeneous scenarios.