# HeteFedRec: Federated Recommender Systems with Model Heterogeneity

arXiv: https://arxiv.org/abs/2307.12810 · HTML: https://mlsanity.com/paper/2307.12810/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2023.

## Built on by (most cited first, 20 of 20)

- [OpenFGL: A Comprehensive Benchmark for Federated Graph Learning](https://mlsanity.com/paper/2408.16288/) (2024)
- [Federated Distillation: A Survey](https://mlsanity.com/paper/2404.08564/) (2024)
- [Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach](https://mlsanity.com/paper/2408.08931/) (2024)
- [Co-clustering for Federated Recommender System](https://mlsanity.com/paper/2411.01690/) (2024)
- [A Comprehensive Data-centric Overview of Federated Graph Learning](https://mlsanity.com/paper/2507.16541/) (2025)
- [Tackling Data Heterogeneity in Federated Time Series Forecasting](https://mlsanity.com/paper/2411.15716/) (2024)
- [FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures](https://mlsanity.com/paper/2505.06497/) (2025)
- [Efficient Federated Learning with Timely Update Dissemination](https://mlsanity.com/paper/2507.06031/) (2025)
- [Federated Recommender System with Data Valuation for E-commerce Platform](https://mlsanity.com/paper/2509.11196/) (2025)
- [Federated Vision-Language-Recommendation with Personalized Fusion](https://mlsanity.com/paper/2410.08478/) (2024)
- [UFGraphFR: Graph Federation Recommendation System based on User Text description features](https://mlsanity.com/paper/2501.08044/) (2025)
- [Brain Storm Optimization Based Swarm Learning for Diabetic Retinopathy Image Classification](https://mlsanity.com/paper/2404.15585/) (2024)
- [A Scenario-Oriented Survey of Federated Recommender Systems: Techniques, Challenges, and Future Directions](https://mlsanity.com/paper/2508.19620/) (2025)
- [MDiffFR: Modality-Guided Diffusion Generation for Cold-start Items in Federated Recommendation](https://mlsanity.com/paper/2512.24715/) (2025)
- [On-Device Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2601.09306/) (2026)
- [LLM-driven Multimodal Recommendation](https://mlsanity.com/paper/2602.05474/) (2026)
- [SDFed: Bridging Local Global Discrepancy via Subspace Refinement and Divergence Control in Federated Prompt Learning](https://mlsanity.com/paper/2602.08590/) (2026)
- [Empowering Contrastive Federated Sequential Recommendation with LLMs](https://mlsanity.com/paper/2602.09306/) (2026)
- [Improving Federated Graph Recommendation with Semantic Guidance](https://mlsanity.com/paper/2606.15277/) (2026)
- [FedTopo: Relation-Level Topology Sharing for Model-Heterogeneous Federated Learning](https://mlsanity.com/paper/2607.26801/) (2026)

## Abstract (arXiv)

Owing to the nature of privacy protection, federated recommender systems (FedRecs) have garnered increasing interest in the realm of on-device recommender systems. However, most existing FedRecs only allow participating clients to collaboratively train a recommendation model of the same public parameter size. Training a model of the same size for all clients can lead to suboptimal performance since clients possess varying resources. For example, clients with limited training data may prefer to train a smaller recommendation model to avoid excessive data consumption, while clients with sufficient data would benefit from a larger model to achieve higher recommendation accuracy. To address the above challenge, this paper introduces HeteFedRec, a novel FedRec framework that enables the assignment of personalized model sizes to participants. In HeteFedRec, we present a heterogeneous recommendation model aggregation strategy, including a unified dual-task learning mechanism and a dimensional decorrelation regularization, to allow knowledge aggregation among recommender models of different sizes. Additionally, a relation-based ensemble knowledge distillation method is proposed to effectively distil knowledge from heterogeneous item embeddings. Extensive experiments conducted on three real-world recommendation datasets demonstrate the effectiveness and efficiency of HeteFedRec in training federated recommender systems under heterogeneous settings.