TinyProto: Communication-Efficient Federated Learning with Sparse Prototypes in Resource-Constrained Environments
Not on the map. Why?
Abstract (arXiv)
Communication efficiency in federated learning (FL) remains a critical challenge for resource-constrained environments. While prototype-based FL reduces communication overhead by sharing class prototypes-mean activations in the penultimate layer-instead of model parameters, its efficiency decreases with larger feature dimensions and class counts. We propose TinyProto, which addresses these limitations through Class-wise Prototype Sparsification (CPS) and adaptive prototype scaling. CPS enables structured sparsity by allocating specific dimensions to class prototypes and transmitting only non-zero elements, while adaptive scaling adjusts prototypes based on class distributions. Our experiments show TinyProto reduces communication costs by up to 4x compared to existing methods while maintaining performance. Beyond its communication efficiency, TinyProto offers crucial advantages: achieving compression without client-side computational overhead and supporting heterogeneous architectures, making it ideal for resource-constrained heterogeneous FL.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 39 references that are in the map. First posted on arXiv 6 Jul 2025.
Builds on
28 of 39 references.
References · 28
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2014Going Deeper with Convolutionscited by 4,241
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
- 2017A Closer Look at Memorization in Deep Networkscited by 942
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2017Deep Mutual Learningcited by 628
- 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
16 more
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
- 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
- 2018Sparsified SGD with Memorycited by 376
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
- 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 252
- 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
- 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
- 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
- 2017Generalization in Deep Learningcited by 146
- 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 81
- 2018Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGDcited by 77
- 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
- 2022FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Servercited by 29
- 2024Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decompositioncited by 15
Built on by
Not cited yet.