# Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation

arXiv: https://arxiv.org/abs/2210.04497 · HTML: https://mlsanity.com/paper/2210.04497/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 29 of 29)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models](https://mlsanity.com/paper/2007.06778/) (2020)
- [End-to-End Bias Mitigation by Modelling Biases in Corpora](https://mlsanity.com/paper/1909.06321/) (2019)
- [Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness](https://mlsanity.com/paper/2010.08001/) (2020)
- [Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual](https://mlsanity.com/paper/1908.10763/) (2019)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [Did the Model Understand the Question?](https://mlsanity.com/paper/1805.05492/) (2018)
- [Automatic Shortcut Removal for Self-Supervised Representation Learning](https://mlsanity.com/paper/2002.08822/) (2020)
- [Why Machine Reading Comprehension Models Learn Shortcuts?](https://mlsanity.com/paper/2106.01024/) (2021)
- [Curriculum-Meta Learning for Order-Robust Continual Relation Extraction](https://mlsanity.com/paper/2101.01926/) (2021)
- [Classifying Relations by Ranking with Convolutional Neural Networks](https://mlsanity.com/paper/1504.06580/) (2015)
- [Consistent Representation Learning for Continual Relation Extraction](https://mlsanity.com/paper/2203.02721/) (2022)
- [Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation](https://mlsanity.com/paper/1601.03651/) (2016)
- [Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling](https://mlsanity.com/paper/1506.07650/) (2015)
- [Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification](https://mlsanity.com/paper/2108.12844/) (2021)
- [Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach](https://mlsanity.com/paper/2209.00243/) (2022)
- [Contextualised Graph Attention for Improved Relation Extraction](https://mlsanity.com/paper/2004.10624/) (2020)

## Built on by (most cited first, 13 of 13)

- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Towards Lifelong Learning of Large Language Models: A Survey](https://mlsanity.com/paper/2406.06391/) (2024)
- [Rationale-Enhanced Language Models are Better Continual Relation Learners](https://mlsanity.com/paper/2310.06547/) (2023)
- [Improving Continual Relation Extraction by Distinguishing Analogous Semantics](https://mlsanity.com/paper/2305.06620/) (2023)
- [Enhancing Continual Relation Extraction via Classifier Decomposition](https://mlsanity.com/paper/2305.04636/) (2023)
- [InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective](https://mlsanity.com/paper/2310.06362/) (2023)
- [Learning to Learn for Few-shot Continual Active Learning](https://mlsanity.com/paper/2311.03732/) (2023)
- [SANTA: Separate Strategies for Inaccurate and Incomplete Annotation Noise in Distantly-Supervised Named Entity Recognition](https://mlsanity.com/paper/2305.04076/) (2023)
- [Making Pre-trained Language Models Better Continual Few-Shot Relation Extractors](https://mlsanity.com/paper/2402.15713/) (2024)
- [Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild](https://mlsanity.com/paper/2305.07085/) (2023)
- [Learning Wisdom from Errors: Promoting LLM's Continual Relation Learning through Exploiting Error Cases](https://mlsanity.com/paper/2508.12031/) (2025)
- [DP-CRE: Continual Relation Extraction via Decoupled Contrastive Learning and Memory Structure Preservation](https://mlsanity.com/paper/2403.02718/) (2024)
- [RepCL: Exploring Effective Representation for Continual Text Classification](https://mlsanity.com/paper/2305.07289/) (2023)

## Abstract (arXiv)

Continual relation extraction (CRE) aims to continually learn new relations from a class-incremental data stream. CRE model usually suffers from catastrophic forgetting problem, i.e., the performance of old relations seriously degrades when the model learns new relations. Most previous work attributes catastrophic forgetting to the corruption of the learned representations as new relations come, with an implicit assumption that the CRE models have adequately learned the old relations. In this paper, through empirical studies we argue that this assumption may not hold, and an important reason for catastrophic forgetting is that the learned representations do not have good robustness against the appearance of analogous relations in the subsequent learning process. To address this issue, we encourage the model to learn more precise and robust representations through a simple yet effective adversarial class augmentation mechanism (ACA), which is easy to implement and model-agnostic. Experimental results show that ACA can consistently improve the performance of state-of-the-art CRE models on two popular benchmarks.