# Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization

arXiv: https://arxiv.org/abs/2304.11524 · HTML: https://mlsanity.com/paper/2304.11524/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 37 references that are in the map.
- First posted on arXiv 23 Apr 2023.

## Builds on (most cited first, 30 of 31)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://mlsanity.com/paper/2105.00243/) (2021)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2109.09784/) (2021)
- [Federated Learning with Label Distribution Skew via Logits Calibration](https://mlsanity.com/paper/2209.00189/) (2022)
- [CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation](https://mlsanity.com/paper/2109.05729/) (2021)
- [The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/2109.02869/) (2021)
- [pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning](https://mlsanity.com/paper/2206.03655/) (2022)
- [Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization](https://mlsanity.com/paper/2205.06009/) (2022)
- [Pretrained Models for Multilingual Federated Learning](https://mlsanity.com/paper/2206.02291/) (2022)
- [Summarizing Patients Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models](https://mlsanity.com/paper/2208.08408/) (2022)
- [Federated Learning with Position-Aware Neurons](https://mlsanity.com/paper/2203.14666/) (2022)
- [ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation](https://mlsanity.com/paper/2106.01597/) (2021)
- [Personalized Federated Learning with Contextualized Generalization](https://mlsanity.com/paper/2106.13044/) (2021)
- [Enriching and Controlling Global Semantics for Text Summarization](https://mlsanity.com/paper/2109.10616/) (2021)
- [Federated Learning Challenges and Opportunities: An Outlook](https://mlsanity.com/paper/2202.00807/) (2022)
- [A Quantitative Metric for Privacy Leakage in Federated Learning](https://mlsanity.com/paper/2102.13472/) (2021)
- [Training Mixed-Domain Translation Models via Federated Learning](https://mlsanity.com/paper/2205.01557/) (2022)
- [Federated Split BERT for Heterogeneous Text Classification](https://mlsanity.com/paper/2205.13299/) (2022)
- [A Secure and Efficient Federated Learning Framework for NLP](https://mlsanity.com/paper/2201.11934/) (2022)
- [ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization](https://mlsanity.com/paper/2201.05313/) (2022)
- [A Privacy-Preserving Subgraph-Level Federated Graph Neural Network via Differential Privacy](https://mlsanity.com/paper/2206.03492/) (2022)
- [Parallel Interactive Networks for Multi-Domain Dialogue State Generation](https://mlsanity.com/paper/2009.07616/) (2020)
- [A Fair Federated Learning Framework With Reinforcement Learning](https://mlsanity.com/paper/2205.13415/) (2022)
- [Mitigating Data Scarceness through Data Synthesis, Augmentation and Curriculum for Abstractive Summarization](https://mlsanity.com/paper/2109.08569/) (2021)
- [Modular Federated Learning](https://mlsanity.com/paper/2209.03090/) (2022)

## Built on by (most cited first, 1 of 1)

- [FedMCP: Parameter-Efficient Federated Learning with Model-Contrastive Personalization](https://mlsanity.com/paper/2409.00116/) (2024)

## Abstract (arXiv)

Text summarization is essential for information aggregation and demands large amounts of training data. However, concerns about data privacy and security limit data collection and model training. To eliminate this concern, we propose a federated learning text summarization scheme, which allows users to share the global model in a cooperative learning manner without sharing raw data. Personalized federated learning (PFL) balances personalization and generalization in the process of optimizing the global model, to guide the training of local models. However, multiple local data have different distributions of semantics and context, which may cause the local model to learn deviated semantic and context information. In this paper, we propose FedSUMM, a dynamic gradient adapter to provide more appropriate local parameters for local model. Simultaneously, FedSUMM uses differential privacy to prevent parameter leakage during distributed training. Experimental evidence verifies FedSUMM can achieve faster model convergence on PFL algorithm for task-specific text summarization, and the method achieves superior performance for different optimization metrics for text summarization.