# Label Granularity Skew in Federated Learning with Hierarchical Image Classification

arXiv: https://arxiv.org/abs/2608.09236 · HTML: https://mlsanity.com/paper/2608.09236/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Progressive Identification of True Labels for Partial-Label Learning](https://mlsanity.com/paper/2002.08053/) (2020)
- [Open Vocabulary Scene Parsing](https://mlsanity.com/paper/1703.08769/) (2017)
- [Hierarchical Image Classification using Entailment Cone Embeddings](https://mlsanity.com/paper/2004.03459/) (2020)
- [FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity](https://mlsanity.com/paper/2305.05230/) (2023)
- [Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification](https://mlsanity.com/paper/2201.03194/) (2022)
- [Integrating domain knowledge: using hierarchies to improve deep classifiers](https://mlsanity.com/paper/1811.07125/) (2018)
- [Hierarchical classification at multiple operating points](https://mlsanity.com/paper/2210.10929/) (2022)
- [Federated Learning with Instance-Dependent Noisy Label](https://mlsanity.com/paper/2312.10324/) (2023)
- [FedMLP: Federated Multi-Label Medical Image Classification under Task Heterogeneity](https://mlsanity.com/paper/2406.18995/) (2024)

## Abstract (arXiv)

Federated learning enables privacy-preserving collaboration across distributed devices without centralizing local data. However, clients may differ not only in data distributions but also in domain knowledge and annotation capabilities. In this paper, we introduce label granularity skew, a new form of statistical heterogeneity in federated hierarchical classification, in which clients provide taxonomy-consistent labels at different levels of detail within a shared class hierarchy. To model this heterogeneity, we generate client-specific local label hierarchies using a probabilistic relational neighbor classifier and construct a WordNet-guided hierarchy via silhouette score-based coarsening. Our analysis shows that strongly coupled hierarchical models are sensitive to incomplete supervision, while the conditional softmax classifier is more robust. Based on this insight, we propose Branch-wise Decoupled Fine-Tuning (BDFT) and its federated version, FedBDFT, which fine-tune branch-wise classifiers and aggregate them through federated optimization. Experiments on CIFAR-100, TinyImageNet, and ImageNet show that FedBDFT substantially improves robustness under severe label granularity skew, with average gains of 27.9% and 56.4% at skewness levels of 0.6 and 0.9, respectively. Zero-shot results further indicate that FedBDFT better preserves hierarchical representations for unseen fine-grained classes. These findings demonstrate its effectiveness for federated hierarchical classification with heterogeneous label granularities.