# Causal Intervention for Abstractive Related Work Generation

arXiv: https://arxiv.org/abs/2305.13685 · HTML: https://mlsanity.com/paper/2305.13685/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://mlsanity.com/paper/1907.12461/) (2019)
- [Leveraging Graph to Improve Abstractive Multi-Document Summarization](https://mlsanity.com/paper/2005.10043/) (2020)
- [Identifying Spurious Correlations for Robust Text Classification](https://mlsanity.com/paper/2010.02458/) (2020)
- [Target-aware Abstractive Related Work Generation with Contrastive Learning](https://mlsanity.com/paper/2205.13339/) (2022)
- [Empowering Language Understanding with Counterfactual Reasoning](https://mlsanity.com/paper/2106.03046/) (2021)
- [Automatic Related Work Generation: A Meta Study](https://mlsanity.com/paper/2201.01880/) (2022)
- [Differentiable N-gram Objective on Abstractive Summarization](https://mlsanity.com/paper/2202.04003/) (2022)
- [An Ion Exchange Mechanism Inspired Story Ending Generator for Different Characters](https://mlsanity.com/paper/2209.00200/) (2022)

## Built on by (most cited first, 5 of 5)

- [Related Work and Citation Text Generation: A Survey](https://mlsanity.com/paper/2404.11588/) (2024)
- [OARelatedWork: A Large-Scale Dataset of Related Work Sections with Full-texts from Open Access Sources](https://mlsanity.com/paper/2405.01930/) (2024)
- [Expert Preference-based Evaluation of Automated Related Work Generation](https://mlsanity.com/paper/2508.07955/) (2025)
- [Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation](https://mlsanity.com/paper/2505.19647/) (2025)
- [Modelling and Classifying the Components of a Literature Review](https://mlsanity.com/paper/2508.04337/) (2025)

## Abstract (arXiv)

Abstractive related work generation has attracted increasing attention in generating coherent related work that better helps readers grasp the background in the current research. However, most existing abstractive models ignore the inherent causality of related work generation, leading to low quality of generated related work and spurious correlations that affect the models' generalizability. In this study, we argue that causal intervention can address these limitations and improve the quality and coherence of the generated related works. To this end, we propose a novel Causal Intervention Module for Related Work Generation (CaM) to effectively capture causalities in the generation process and improve the quality and coherence of the generated related works. Specifically, we first model the relations among sentence order, document relation, and transitional content in related work generation using a causal graph. Then, to implement the causal intervention and mitigate the negative impact of spurious correlations, we use do-calculus to derive ordinary conditional probabilities and identify causal effects through CaM. Finally, we subtly fuse CaM with Transformer to obtain an end-to-end generation model. Extensive experiments on two real-world datasets show that causal interventions in CaM can effectively promote the model to learn causal relations and produce related work of higher quality and coherence.