# A Secure and Efficient Federated Learning Framework for NLP

arXiv: https://arxiv.org/abs/2201.11934 · HTML: https://mlsanity.com/paper/2201.11934/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Differentially Private Federated Learning: A Client Level Perspective](https://mlsanity.com/paper/1712.07557/) (2017)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [A Hybrid Approach to Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1812.03224/) (2018)
- [Model Compression with Adversarial Robustness: A Unified Optimization Framework](https://mlsanity.com/paper/1902.03538/) (2019)
- [CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices](https://mlsanity.com/paper/1708.08917/) (2017)
- [PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices](https://mlsanity.com/paper/1909.05073/) (2019)
- [Against Membership Inference Attack: Pruning is All You Need](https://mlsanity.com/paper/2008.13578/) (2020)
- [SAPAG: A Self-Adaptive Privacy Attack From Gradients](https://mlsanity.com/paper/2009.06228/) (2020)
- [Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank](https://mlsanity.com/paper/1703.00144/) (2017)
- [DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks](https://mlsanity.com/paper/1911.08020/) (2019)
- [ESMFL: Efficient and Secure Models for Federated Learning](https://mlsanity.com/paper/2009.01867/) (2020)

## Built on by (most cited first, 9 of 9)

- [FedAdapter: Efficient Federated Learning for Modern NLP](https://mlsanity.com/paper/2205.10162/) (2022)
- [Delving into the Adversarial Robustness of Federated Learning](https://mlsanity.com/paper/2302.09479/) (2023)
- [Federated Few-Shot Learning for Mobile NLP](https://mlsanity.com/paper/2212.05974/) (2022)
- [LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classification](https://mlsanity.com/paper/2402.16515/) (2024)
- [COALA: A Practical and Vision-Centric Federated Learning Platform](https://mlsanity.com/paper/2407.16560/) (2024)
- [Keep It Simple: Fault Tolerance Evaluation of Federated Learning with Unreliable Clients](https://mlsanity.com/paper/2305.09856/) (2023)
- [DSFL: A Dual-Server Byzantine-Resilient Federated Learning Framework via Group-Based Secure Aggregation](https://mlsanity.com/paper/2509.08449/) (2025)
- [Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization](https://mlsanity.com/paper/2304.11524/) (2023)
- [Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs](https://mlsanity.com/paper/2605.10748/) (2026)

## Abstract (arXiv)

In this work, we consider the problem of designing secure and efficient federated learning (FL) frameworks. Existing solutions either involve a trusted aggregator or require heavyweight cryptographic primitives, which degrades performance significantly. Moreover, many existing secure FL designs work only under the restrictive assumption that none of the clients can be dropped out from the training protocol. To tackle these problems, we propose SEFL, a secure and efficient FL framework that (1) eliminates the need for the trusted entities; (2) achieves similar and even better model accuracy compared with existing FL designs; (3) is resilient to client dropouts. Through extensive experimental studies on natural language processing (NLP) tasks, we demonstrate that the SEFL achieves comparable accuracy compared to existing FL solutions, and the proposed pruning technique can improve runtime performance up to 13.7x.