# Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective

arXiv: https://arxiv.org/abs/2412.08285 · HTML: https://mlsanity.com/paper/2412.08285/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality](https://mlsanity.com/paper/2310.07234/) (2023)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [Consistent Representation Learning for Continual Relation Extraction](https://mlsanity.com/paper/2203.02721/) (2022)
- [Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts](https://mlsanity.com/paper/2410.02200/) (2024)
- [Improving Continual Relation Extraction by Distinguishing Analogous Semantics](https://mlsanity.com/paper/2305.06620/) (2023)
- [Preserving Generalization of Language models in Few-shot Continual Relation Extraction](https://mlsanity.com/paper/2410.00334/) (2024)
- [Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning](https://mlsanity.com/paper/2410.04327/) (2024)

## Built on by (most cited first, 6 of 6)

- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts](https://mlsanity.com/paper/2501.18936/) (2025)
- [Few-Shot, No Problem: Descriptive Continual Relation Extraction](https://mlsanity.com/paper/2502.20596/) (2025)
- [One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning](https://mlsanity.com/paper/2509.24483/) (2025)
- [WAVE++: Capturing Within-Task Variance for Continual Relation Extraction with Adaptive Prompting](https://mlsanity.com/paper/2505.13944/) (2025)
- [LEAF: A Robust Expert-Based Framework for Few-Shot Continual Event Detection](https://mlsanity.com/paper/2509.24547/) (2025)

## Abstract (arXiv)

To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix-tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction.