mlsanityData to 14 Oct 2026

Federated Incremental Named Entity Recognition

arXiv 2411.11623 ↗First posted 18 Nov 2024cs.CLcited by 2

On the map: Knowledge distillation → Federated, robust and private distillation

Abstract (arXiv)

Federated Named Entity Recognition (FNER) boosts model training within each local client by aggregating the model updates of decentralized local clients, without sharing their private data. However, existing FNER methods assume fixed entity types and local clients in advance, leading to their ineffectiveness in practical applications. In a more realistic scenario, local clients receive new entity types continuously, while new local clients collecting novel data may irregularly join the global FNER training. This challenging setup, referred to here as Federated Incremental NER, renders the global model suffering from heterogeneous forgetting of old entity types from both intra-client and inter-client perspectives. To overcome these challenges, we propose a Local-Global Forgetting Defense (LGFD) model. Specifically, to address intra-client forgetting, we develop a structural knowledge distillation loss to retain the latent space's feature structure and a pseudo-label-guided inter-type contrastive loss to enhance discriminative capability over different entity types, effectively preserving previously learned knowledge within local clients. To tackle inter-client forgetting, we propose a task switching monitor that can automatically identify new entity types under privacy protection and store the latest old global model for knowledge distillation and pseudo-labeling. Experiments demonstrate significant improvement of our LGFD model over comparison methods.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation. Builds on 35 of its 71 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

35 of 71 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  8. 2023PaLM 2 Technical Reportcited by 1,162
  9. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  10. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  11. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  12. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
23 more
  1. 2020Federated Learning with Matched Averagingcited by 564
  2. 2016Neural Architectures for Named Entity Recognitioncited by 550
  3. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  4. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  5. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  6. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 282
  7. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  9. 2022Federated Class-Incremental Learningcited by 127
  10. 2019A continual learning survey: Defying forgetting in classification taskscited by 118
  11. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 100
  12. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
  13. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  14. 2023Federated Incremental Semantic Segmentationcited by 36
  15. 2024How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?cited by 33
  16. 2020FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learningcited by 24
  17. 2023Subspace Distillation for Continual Learningcited by 12
  18. 2023Continual Named Entity Recognition without Catastrophic Forgettingcited by 11
  19. 2022Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognitioncited by 10
  20. 2023Task Relation Distillation and Prototypical Pseudo Label for Incremental Named Entity Recognitioncited by 9
  21. 2022Learning "O" Helps for Learning More: Handling the Concealed Entity Problem for Class-incremental NERcited by 8
  22. 2024Incremental Sequence Labeling: A Tale of Two Shiftscited by 5
  23. 2022Federated Named Entity Recognitioncited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Exploring Stability-Plasticity Trade-offs for Continual Named Entity Recognitioncited by 1
  2. 2025Information-Theoretic Complementary Prompts for Improved Continual Text Classificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning
  2. 2023On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions
  3. 2024Federated Class-Incremental Learning with New-Class Augmented Self-Distillation
  4. 2024Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
  5. 2022Unintended Memorization and Timing Attacks in Named Entity Recognition Models
  6. 2023A Neural Span-Based Continual Named Entity Recognition Model
  7. 2022Closing the Gap between Client and Global Model Performance in Heterogeneous Federated Learning
  8. 2024Forgetting Through Transforming: Enabling Federated Unlearning via Class-Aware Representation Transformation