mlsanityData to 14 Oct 2026

CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learning

arXiv 2609.05884 ↗First posted 5 Sep 2026cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Conventional federated learning relies on parameter averaging, which forces clients to be doubly homogeneous: all must run an identical architecture, and accuracy degrades when local data are non-IID. Decentralized federated distillation sidesteps both: each client runs its peers' model snapshots as teachers on its own local data and distills from their soft predictions, with no server, no public data, and no shared architecture. Under severe non-IID skew, however, the trustworthiness of the aggregated teacher target is a matter of degree, yet existing pipelines make hard, all-or-nothing decisions: outlier teachers are discarded by threshold, and whatever target survives is trusted in full. We propose CALM, which replaces every hard decision with a smooth trust gate at three levels: per class, teachers are weighted by agreement with the peer consensus; per sample, distillation is scaled by the teachers' divergence from that target; and a label gate scales it by how strongly the target supports the sample's true label. None of this adds communication or auxiliary data. On CIFAR-10, SVHN, OrganAMNIST, and Google Speech Commands with heterogeneous client architectures under Dirichlet label skew, CALM consistently outperforms uniform and hard-filtered distillation and matches or exceeds competing heterogeneous-FL methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 22 references that are in the map. First posted on arXiv 5 Sep 2026.

Builds on

17 of 22 references.

References · 17

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  11. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
  12. 2021Confidence-Aware Multi-Teacher Knowledge Distillationcited by 39
5 more
  1. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21
  2. 2023A Survey on Model-heterogeneous Federated Learning: Problems, Methods, and Prospectscited by 9
  3. 2025A Thorough Assessment of the Non-IID Data Impact in Federated Learningcited by 5
  4. 2024DFML: Decentralized Federated Mutual Learningcited by 4
  5. 2026CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning
  2. 2024Exploring Vacant Classes in Label-Skewed Federated Learning
  3. 2026FedeKD: Energy-Based Gating for Robust Federated Knowledge Distillation under Heterogeneous Settings
  4. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning
  5. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillation
  6. 2025OvA-LP: A Simple and Efficient Framework for Federated Learning on Non-IID Data
  7. 2026Certified Unlearning in Decentralized Federated Learning
  8. 2023Federated Learning on Non-iid Data via Local and Global Distillation