# Fed-Focal Loss for imbalanced data classification in Federated Learning

arXiv: https://arxiv.org/abs/2011.06283 · HTML: https://mlsanity.com/paper/2011.06283/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 26 of 26)

- [Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features](https://mlsanity.com/paper/2204.13399/) (2022)
- [Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting](https://mlsanity.com/paper/2109.07258/) (2021)
- [FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillation](https://mlsanity.com/paper/2205.00172/) (2022)
- [FedIIC: Towards Robust Federated Learning for Class-Imbalanced Medical Image Classification](https://mlsanity.com/paper/2206.13803/) (2022)
- [A Survey on Class Imbalance in Federated Learning](https://mlsanity.com/paper/2303.11673/) (2023)
- [CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2312.08648/) (2023)
- [An Experimental Study of Class Imbalance in Federated Learning](https://mlsanity.com/paper/2109.04094/) (2021)
- [FACMIC: Federated Adaptative CLIP Model for Medical Image Classification](https://mlsanity.com/paper/2410.14707/) (2024)
- [FLSys: Toward an Open Ecosystem for Federated Learning Mobile Apps](https://mlsanity.com/paper/2111.09445/) (2021)
- [Federated Test-Time Adaptive Face Presentation Attack Detection with Dual-Phase Privacy Preservation](https://mlsanity.com/paper/2110.12613/) (2021)
- [Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning](https://mlsanity.com/paper/2301.10394/) (2023)
- [MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning](https://mlsanity.com/paper/2409.06067/) (2024)
- [Demystifying the Effects of Non-Independence in Federated Learning](https://mlsanity.com/paper/2103.11226/) (2021)
- [Improving the Robustness of Federated Learning for Severely Imbalanced Datasets](https://mlsanity.com/paper/2204.13414/) (2022)
- [Decoupled Federated Learning on Long-Tailed and Non-IID data with Feature Statistics](https://mlsanity.com/paper/2403.08364/) (2024)
- [CAPT: Class-Aware Prompt Tuning for Federated Long-Tailed Learning with Vision-Language Model](https://mlsanity.com/paper/2503.06993/) (2025)
- [SAFE: Self-Adjustment Federated Learning Framework for Remote Sensing Collaborative Perception](https://mlsanity.com/paper/2504.03700/) (2025)
- [Towards Unbiased Federated Graph Learning: Label and Topology Perspectives](https://mlsanity.com/paper/2504.09963/) (2025)
- [Federated CLIP for Resource-Efficient Heterogeneous Medical Image Classification](https://mlsanity.com/paper/2511.07929/) (2025)
- [pFedBBN: A Personalized Federated Test-Time Adaptation with Balanced Batch Normalization for Class-Imbalanced Data](https://mlsanity.com/paper/2511.18066/) (2025)
- [Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning](https://mlsanity.com/paper/2605.02247/) (2026)
- [FedProf: Selective Federated Learning with Representation Profiling](https://mlsanity.com/paper/2102.01733/) (2021)
- [A Novel Plagiarism Detection Approach Combining BERT-based Word Embedding, Attention-based LSTMs and an Improved Differential Evolution Algorithm](https://mlsanity.com/paper/2305.02374/) (2023)
- [A cautionary tale on the cost-effectiveness of collaborative AI in real-world medical applications](https://mlsanity.com/paper/2412.06494/) (2024)
- [FedSM: Robust Semantics-Guided Feature Mixup for Bias Reduction in Federated Learning with Long-Tail Data](https://mlsanity.com/paper/2510.27240/) (2025)
- [Federated Vision Transformer with Adaptive Focal Loss for Medical Image Classification](https://mlsanity.com/paper/2602.01633/) (2026)

## Abstract (arXiv)

The Federated Learning setting has a central server coordinating the training of a model on a network of devices. One of the challenges is variable training performance when the dataset has a class imbalance. In this paper, we address this by introducing a new loss function called Fed-Focal Loss. We propose to address the class imbalance by reshaping cross-entropy loss such that it down-weights the loss assigned to well-classified examples along the lines of focal loss. Additionally, by leveraging a tunable sampling framework, we take into account selective client model contributions on the central server to further focus the detector during training and hence improve its robustness. Using a detailed experimental analysis with the VIRTUAL (Variational Federated Multi-Task Learning) approach, we demonstrate consistently superior performance in both the balanced and unbalanced scenarios for MNIST, FEMNIST, VSN and HAR benchmarks. We obtain a more than 9% (absolute percentage) improvement in the unbalanced MNIST benchmark. We further show that our technique can be adopted across multiple Federated Learning algorithms to get improvements.