mlsanityData to 14 Oct 2026

Benchmarking Mutual Information-based Loss Functions in Federated Learning

arXiv 2504.11877 ↗First posted 16 Apr 2025cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has attracted considerable interest due to growing privacy concerns and regulations like the General Data Protection Regulation (GDPR), which stresses the importance of privacy-preserving and fair machine learning approaches. In FL, model training takes place on decentralized data, so as to allow clients to upload a locally trained model and receive a globally aggregated model without exposing sensitive information. However, challenges related to fairness-such as biases, uneven performance among clients, and the "free rider" issue complicates its adoption. In this paper, we examine the use of Mutual Information (MI)-based loss functions to address these concerns. MI has proven to be a powerful method for measuring dependencies between variables and optimizing deep learning models. By leveraging MI to extract essential features and minimize biases, we aim to improve both the fairness and effectiveness of FL systems. Through extensive benchmarking, we assess the impact of MI-based losses in reducing disparities among clients while enhancing the overall performance of FL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 45 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

20 of 45 references.

References · 20

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  8. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  9. 2018MINE: Mutual Information Neural Estimationcited by 683
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2019On Variational Bounds of Mutual Informationcited by 559
  12. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
8 more
  1. 2019Agnostic Federated Learningcited by 456
  2. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  3. 2019Fair Resource Allocation in Federated Learningcited by 394
  4. 2020Free-rider Attacks on Model Aggregation in Federated Learningcited by 57
  5. 2021Resource-Efficient Federated Learningcited by 26
  6. 2023Bias Propagation in Federated Learningcited by 12
  7. 2020Combating the Instability of Mutual Information-based Losses via Regularizationcited by 5
  8. 2024Federated Fairness Analytics: Quantifying Fairness in Federated Learningcited by 4

Built on by

Not cited yet.