# Rationale-Enhanced Language Models are Better Continual Relation Learners

arXiv: https://arxiv.org/abs/2310.06547 · HTML: https://mlsanity.com/paper/2310.06547/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Explanations from Large Language Models Make Small Reasoners Better](https://mlsanity.com/paper/2210.06726/) (2022)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [An Empirical Study on Information Extraction using Large Language Models](https://mlsanity.com/paper/2305.14450/) (2023)
- [Can Rationalization Improve Robustness?](https://mlsanity.com/paper/2204.11790/) (2022)
- [Consistent Representation Learning for Continual Relation Extraction](https://mlsanity.com/paper/2203.02721/) (2022)
- [Improving Continual Relation Extraction through Prototypical Contrastive Learning](https://mlsanity.com/paper/2210.04513/) (2022)
- [Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation](https://mlsanity.com/paper/2210.04497/) (2022)
- [Improving Continual Relation Extraction by Distinguishing Analogous Semantics](https://mlsanity.com/paper/2305.06620/) (2023)
- [Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach](https://mlsanity.com/paper/2209.00243/) (2022)

## Built on by (most cited first, 13 of 13)

- [Continual Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2402.01364/) (2024)
- [Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models](https://mlsanity.com/paper/2405.15574/) (2024)
- [SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs](https://mlsanity.com/paper/2509.20758/) (2025)
- [Retrieval-Augmented Generation-based Relation Extraction](https://mlsanity.com/paper/2404.13397/) (2024)
- [Relation Extraction with Fine-Tuned Large Language Models in Retrieval Augmented Generation Frameworks](https://mlsanity.com/paper/2406.14745/) (2024)
- [Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation](https://mlsanity.com/paper/2410.13944/) (2024)
- [ADEPT: Continual Pretraining via Adaptive Expansion and Dynamic Decoupled Tuning](https://mlsanity.com/paper/2510.10071/) (2025)
- [Learning Wisdom from Errors: Promoting LLM's Continual Relation Learning through Exploiting Error Cases](https://mlsanity.com/paper/2508.12031/) (2025)
- [Leveraging Machine-Generated Rationales to Facilitate Social Meaning Detection in Conversations](https://mlsanity.com/paper/2406.19545/) (2024)
- [Lifelong Event Detection via Optimal Transport](https://mlsanity.com/paper/2410.08905/) (2024)
- [WAVE++: Capturing Within-Task Variance for Continual Relation Extraction with Adaptive Prompting](https://mlsanity.com/paper/2505.13944/) (2025)
- [When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms](https://mlsanity.com/paper/2604.16767/) (2026)
- [Post-Training Language Models for Continual Relation Extraction](https://mlsanity.com/paper/2504.05214/) (2025)

## Abstract (arXiv)

Continual relation extraction (CRE) aims to solve the problem of catastrophic forgetting when learning a sequence of newly emerging relations. Recent CRE studies have found that catastrophic forgetting arises from the model's lack of robustness against future analogous relations. To address the issue, we introduce rationale, i.e., the explanations of relation classification results generated by large language models (LLM), into CRE task. Specifically, we design the multi-task rationale tuning strategy to help the model learn current relations robustly. We also conduct contrastive rationale replay to further distinguish analogous relations. Experimental results on two standard benchmarks demonstrate that our method outperforms the state-of-the-art CRE models.