mlsanityData to 7 Oct 2026

Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity

arXiv 2206.02633 ↗First posted 30 May 2022cs.IR cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is an effective mechanism for data privacy in recommender systems by running machine learning model training on-device. While prior FL optimizations tackled the data and system heterogeneity challenges faced by FL, they assume the two are independent of each other. This fundamental assumption is not reflective of real-world, large-scale recommender systems -- data and system heterogeneity are tightly intertwined. This paper takes a data-driven approach to show the inter-dependence of data and system heterogeneity in real-world data and quantifies its impact on the overall model quality and fairness. We design a framework, RF^2, to model the inter-dependence and evaluate its impact on state-of-the-art model optimization techniques for federated recommendation tasks. We demonstrate that the impact on fairness can be severe under realistic heterogeneity scenarios, by up to 15.8--41x compared to a simple setup assumed in most (if not all) prior work. It means when realistic system-induced data heterogeneity is not properly modeled, the fairness impact of an optimization can be downplayed by up to 41x. The result shows that modeling realistic system-induced data heterogeneity is essential to achieving fair federated recommendation learning. We plan to open-source RF^2 to enable future design and evaluation of FL innovations.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 50 of its 83 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

50 of 83 references.

References · 50

  1. 2019Advances and Open Problems in Federated Learningcited by 3,021
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  3. 2017Neural Collaborative Filteringcited by 1,108
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  6. 2019Towards Federated Learning at Scale: System Designcited by 826
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2016Wide & Deep Learning for Recommender Systemscited by 745
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  10. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  11. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
38 more
  1. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  2. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  3. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  4. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  5. 2017Deep & Cross Network for Ad Click Predictionscited by 392
  6. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  7. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  8. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 326
  9. 2021A Field Guide to Federated Optimizationcited by 312
  10. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  11. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  12. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  13. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  14. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  15. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  16. 2020Characterising Bias in Compressed Modelscited by 162
  17. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  18. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  19. 2020Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender Systemcited by 94
  20. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  21. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 89
  22. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  23. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  24. 2019Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedbackcited by 79
  25. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  26. 2019Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systemscited by 68
  27. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  28. 2020IBM Federated Learning: an Enterprise Framework White Paper V0.1cited by 57
  29. 2020To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devicescited by 45
  30. 2020Understanding Training Efficiency of Deep Learning Recommendation Models at Scalecited by 43
  31. 2021AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learningcited by 39
  32. 2020Learning to Embed Categorical Features without Embedding Tables for Recommendationcited by 38
  33. 2020Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learningcited by 37
  34. 2021FedJAX: Federated learning simulation with JAXcited by 32
  35. 2021Federated Neural Collaborative Filteringcited by 30
  36. 2021Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfercited by 23
  37. 2020Developing a Recommendation Benchmark for MLPerf Training and Inferencecited by 12
  38. 2019AI Matrix: A Deep Learning Benchmark for Alibaba Data Centerscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 5; 2025: 2; 2026: 1.

17 papers.

Sort:

  1. 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
  2. 2023No prejudice! Fair Federated Graph Neural Networks for Personalized Recommendationcited by 18
  3. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  4. 2023Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devicescited by 7
  5. 2022On Noisy Evaluation in Federated Hyperparameter Tuningcited by 6
  6. 2022FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learningcited by 4
  7. 2024On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representationscited by 4
  8. 2022FedGPO: Heterogeneity-Aware Global Parameter Optimization for Efficient Federated Learningcited by 3
  9. 2023Fairness-aware Differentially Private Collaborative Filteringcited by 1
  10. 2023Exploring and Exploiting Data Heterogeneity in Recommendationcited by 1
  11. 2024HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learningcited by 1
  12. 2025Personalized Recommendation Models in Federated Settings: A Surveycited by 1
5 more
  1. 2023Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspacescited by 0
  2. 2024Towards Fairness in Provably Communication-Efficient Federated Recommender Systemscited by 0
  3. 2024Privacy-Preserving Orthogonal Aggregation for Guaranteeing Gender Fairness in Federated Recommendationcited by 0
  4. 2025A Scenario-Oriented Survey of Federated Recommender Systems: Techniques, Challenges, and Future Directionscited by 0
  5. 2026FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clientscited by 0