mlsanityData to 14 Oct 2026

SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks

arXiv 2505.23801 ↗First posted 26 May 2025cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Background: Federated Learning (FL) has emerged as a promising paradigm for training machine learning models while preserving data privacy. However, applying FL to Natural Language Processing (NLP) tasks presents unique challenges due to semantic heterogeneity across clients, vocabulary mismatches, and varying resource constraints on edge devices. Objectives: This paper introduces SEMFED, a novel semantic-aware resource-efficient federated learning framework specifically designed for heterogeneous NLP tasks. Methods: SEMFED incorporates three key innovations: (1) a semantic-aware client selection mechanism that balances semantic diversity with resource constraints, (2) adaptive NLP-specific model architectures tailored to device capabilities while preserving semantic information, and (3) a communication-efficient semantic feature compression technique that significantly reduces bandwidth requirements. Results: Experimental results on various NLP classification tasks demonstrate that SEMFED achieves an 80.5% reduction in communication costs while maintaining model accuracy above 98%, outperforming state-of-the-art FL approaches. Conclusion: SEMFED effectively manages heterogeneous client environments with varying computational resources, network reliability, and semantic data distributions, making it particularly suitable for real-world federated NLP deployments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 23 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

19 of 23 references.

References · 19

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  6. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  11. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  12. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
7 more
  1. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  2. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  3. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  4. 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 263
  5. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  6. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 85
  7. 2020A Secure Federated Learning Framework for 5G Networkscited by 32

Built on by

Not cited yet.