# Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction arXiv: https://arxiv.org/abs/2305.16559 ยท HTML: https://mlsanity.com/paper/2305.16559/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 50 references that are in the map. - First posted on arXiv 26 May 2023. ## Builds on (most cited first, 28 of 28) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016) - [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017) - [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019) - [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022) - [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023) - [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020) - [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019) - [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021) - [Continual Lifelong Learning in Natural Language Processing: A Survey](https://mlsanity.com/paper/2012.09823/) (2020) - [Few-NERD: A Few-Shot Named Entity Recognition Dataset](https://mlsanity.com/paper/2105.07464/) (2021) - [A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning](https://mlsanity.com/paper/2001.00689/) (2020) - [MAVEN: A Massive General Domain Event Detection Dataset](https://mlsanity.com/paper/2004.13590/) (2020) - [LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5](https://mlsanity.com/paper/2110.07298/) (2021) - [Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning](https://mlsanity.com/paper/2112.02706/) (2021) - [Continual Learning in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2012.15504/) (2020) - [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023) - [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019) - [Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks](https://mlsanity.com/paper/2112.03271/) (2021) - [Continual Prompt Tuning for Dialog State Tracking](https://mlsanity.com/paper/2203.06654/) (2022) - [Curriculum-Meta Learning for Order-Robust Continual Relation Extraction](https://mlsanity.com/paper/2101.01926/) (2021) - [Lifelong Language Knowledge Distillation](https://mlsanity.com/paper/2010.02123/) (2020) - [Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer](https://mlsanity.com/paper/2101.09536/) (2021) - [CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks](https://mlsanity.com/paper/2112.02714/) (2021) - [Consistent Representation Learning for Continual Relation Extraction](https://mlsanity.com/paper/2203.02721/) (2022) - [Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection](https://mlsanity.com/paper/2204.07275/) (2022) - [Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition](https://mlsanity.com/paper/2210.03980/) (2022) - [Hierarchical Inductive Transfer for Continual Dialogue Learning](https://mlsanity.com/paper/2203.10484/) (2022) ## Built on by (most cited first, 9 of 9) - [Towards Lifelong Learning of Large Language Models: A Survey](https://mlsanity.com/paper/2406.06391/) (2024) - [X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspects](https://mlsanity.com/paper/2311.08788/) (2023) - [Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models](https://mlsanity.com/paper/2312.07887/) (2023) - [Modality-Specialized Synergizers for Interleaved Vision-Language Generalists](https://mlsanity.com/paper/2407.03604/) (2024) - [Incremental Sequence Labeling: A Tale of Two Shifts](https://mlsanity.com/paper/2402.10447/) (2024) - [Navigating Ideation Space: Decomposed Conceptual Representations for Positioning Scientific Ideas](https://mlsanity.com/paper/2601.08901/) (2026) - [GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling](https://mlsanity.com/paper/2505.00063/) (2025) - [Can LLMs Learn New Concepts Incrementally without Forgetting?](https://mlsanity.com/paper/2402.08526/) (2024) - [Why Do Neural Networks Forget: A Study of Collapse in Continual Learning](https://mlsanity.com/paper/2603.04580/) (2026) ## Abstract (arXiv) Class-incremental learning (CIL) aims to develop a learning system that can continually learn new classes from a data stream without forgetting previously learned classes. When learning classes incrementally, the classifier must be constantly updated to incorporate new classes, and the drift in decision boundary may lead to severe forgetting. This fundamental challenge, however, has not yet been studied extensively, especially in the setting where no samples from old classes are stored for rehearsal. In this paper, we take a closer look at how the drift in the classifier leads to forgetting, and accordingly, design four simple yet (super-) effective solutions to alleviate the classifier drift: an Individual Classifiers with Frozen Feature Extractor (ICE) framework where we individually train a classifier for each learning session, and its three variants ICE-PL, ICE-O, and ICE-PL&O which further take the logits of previously learned classes from old sessions or a constant logit of an Other class as a constraint to the learning of new classifiers. Extensive experiments and analysis on 6 class-incremental information extraction tasks demonstrate that our solutions, especially ICE-O, consistently show significant improvement over the previous state-of-the-art approaches with up to 44.7% absolute F-score gain, providing a strong baseline and insights for future research on class-incremental learning.