# Device Heterogeneity in Federated Learning: A Superquantile Approach

arXiv: https://arxiv.org/abs/2002.11223 · HTML: https://mlsanity.com/paper/2002.11223/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 21 of 21)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning](https://mlsanity.com/paper/2307.09619/) (2023)
- [Correlated Noise Provably Beats Independent Noise for Differentially Private Learning](https://mlsanity.com/paper/2310.06771/) (2023)
- [Federated Learning with Taskonomy for Non-IID Data](https://mlsanity.com/paper/2103.15947/) (2021)
- [Federated learning: Applications, challenges and future directions](https://mlsanity.com/paper/2205.09513/) (2022)
- [Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond](https://mlsanity.com/paper/2403.03562/) (2024)
- [Addressing Heterogeneity in Federated Learning via Distributional Transformation](https://mlsanity.com/paper/2210.15025/) (2022)
- [Scaff-PD: Communication Efficient Fair and Robust Federated Learning](https://mlsanity.com/paper/2307.13381/) (2023)
- [A Comprehensive Survey On Client Selections in Federated Learning](https://mlsanity.com/paper/2311.06801/) (2023)
- [Energy-Efficient Channel Decoding for Wireless Federated Learning: Convergence Analysis and Adaptive Design](https://mlsanity.com/paper/2407.13703/) (2024)
- [OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning](https://mlsanity.com/paper/2105.00602/) (2021)
- [Distributionally Robust Clustered Federated Learning: A Case Study in Healthcare](https://mlsanity.com/paper/2410.07039/) (2024)
- [Convergence of Agnostic Federated Averaging](https://mlsanity.com/paper/2507.10325/) (2025)
- [DAG-ACFL: Asynchronous Clustered Federated Learning based on DAG-DLT](https://mlsanity.com/paper/2308.13158/) (2023)
- [Certifiably Robust Model Evaluation in Federated Learning under Meta-Distributional Shifts](https://mlsanity.com/paper/2410.20250/) (2024)
- [Federated Learning with Relative Fairness](https://mlsanity.com/paper/2411.01161/) (2024)
- [Towards Performatively Stable Equilibria in Decision-Dependent Games for Arbitrary Data Distribution Maps](https://mlsanity.com/paper/2509.02619/) (2025)
- [Convergence of empirical subgradients for optimal transport-based objectives](https://mlsanity.com/paper/2605.28134/) (2026)
- [The Privacy-Hallucination Tradeoff in Differentially Private Language Models](https://mlsanity.com/paper/2609.00492/) (2026)
- [FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift](https://mlsanity.com/paper/2609.21057/) (2026)
- [ORDERS: An Empirical Study of Norm-Rank Aggregation for Personalized Federated Learning](https://mlsanity.com/paper/2610.10361/) (2026)

## Abstract (arXiv)

We propose a federated learning framework to handle heterogeneous client devices which do not conform to the population data distribution. The approach hinges upon a parameterized superquantile-based objective, where the parameter ranges over levels of conformity. We present an optimization algorithm and establish its convergence to a stationary point. We show how to practically implement it using secure aggregation by interleaving iterations of the usual federated averaging method with device filtering. We conclude with numerical experiments on neural networks as well as linear models on tasks from computer vision and natural language processing.