# Guided Generation of Cause and Effect

arXiv: https://arxiv.org/abs/2107.09846 · HTML: https://mlsanity.com/paper/2107.09846/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 80 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://mlsanity.com/paper/1704.07138/) (2017)
- [Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks](https://mlsanity.com/paper/1811.01088/) (2018)
- [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://mlsanity.com/paper/1804.06609/) (2018)
- [ASER: A Large-scale Eventuality Knowledge Graph](https://mlsanity.com/paper/1905.00270/) (2019)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [Target-Guided Open-Domain Conversation](https://mlsanity.com/paper/1905.11553/) (2019)
- [Joint Reasoning for Temporal and Causal Relations](https://mlsanity.com/paper/1906.04941/) (2019)
- [Story Ending Generation with Incremental Encoding and Commonsense Knowledge](https://mlsanity.com/paper/1808.10113/) (2018)
- [ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation](https://mlsanity.com/paper/1901.03644/) (2019)
- [Constructing Narrative Event Evolutionary Graph for Script Event Prediction](https://mlsanity.com/paper/1805.05081/) (2018)
- [Automatic Extraction of Causal Relations from Natural Language Texts: A Comprehensive Survey](https://mlsanity.com/paper/1605.07895/) (2016)
- [Sockeye: A Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1712.05690/) (2017)
- [Story Ending Prediction by Transferable BERT](https://mlsanity.com/paper/1905.07504/) (2019)
- [Learning to Rank for Plausible Plausibility](https://mlsanity.com/paper/1906.02079/) (2019)

## Built on by (most cited first, 30 of 33)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Boosting Language Models Reasoning with Chain-of-Knowledge Prompting](https://mlsanity.com/paper/2306.06427/) (2023)
- [Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation](https://mlsanity.com/paper/2305.07375/) (2023)
- [e-CARE: a New Dataset for Exploring Explainable Causal Reasoning](https://mlsanity.com/paper/2205.05849/) (2022)
- [Causal Inference with Large Language Model: A Survey](https://mlsanity.com/paper/2409.09822/) (2024)
- [SciNLI: A Corpus for Natural Language Inference on Scientific Text](https://mlsanity.com/paper/2203.06728/) (2022)
- [Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models](https://mlsanity.com/paper/2210.16433/) (2022)
- [A Critical Review of Causal Reasoning Benchmarks for Large Language Models](https://mlsanity.com/paper/2407.08029/) (2024)
- [Towards Fine-grained Causal Reasoning and QA](https://mlsanity.com/paper/2204.07408/) (2022)
- [UniCausal: Unified Benchmark and Repository for Causal Text Mining](https://mlsanity.com/paper/2208.09163/) (2022)
- [Evaluation of Induced Expert Knowledge in Causal Structure Learning by NOTEARS](https://mlsanity.com/paper/2301.01817/) (2023)
- [NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation](https://mlsanity.com/paper/2312.05979/) (2023)
- [Knowledge-Augmented Language Models for Cause-Effect Relation Classification](https://mlsanity.com/paper/2112.08615/) (2021)
- [CARE: Causality Reasoning for Empathetic Responses by Conditional Graph Generation](https://mlsanity.com/paper/2211.00255/) (2022)
- [A Survey of Methods, Challenges and Perspectives in Causality](https://mlsanity.com/paper/2302.00293/) (2023)
- [Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing](https://mlsanity.com/paper/2305.15338/) (2023)
- [DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination](https://mlsanity.com/paper/2208.09884/) (2022)
- [Causal Question Answering with Reinforcement Learning](https://mlsanity.com/paper/2311.02760/) (2023)
- [InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models](https://mlsanity.com/paper/2312.13503/) (2023)
- [The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning](https://mlsanity.com/paper/2406.19307/) (2024)
- [Narrating Causal Graphs with Large Language Models](https://mlsanity.com/paper/2403.07118/) (2024)
- [A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning](https://mlsanity.com/paper/2406.14164/) (2024)
- [i-Align: an interpretable knowledge graph alignment model](https://mlsanity.com/paper/2308.13755/) (2023)
- [Is ChatGPT the Future of Causal Text Mining? A Comprehensive Evaluation and Analysis](https://mlsanity.com/paper/2402.14484/) (2024)
- [Beyond LLMs: A Linguistic Approach to Causal Graph Generation from Narrative Texts](https://mlsanity.com/paper/2504.07459/) (2025)
- [ReCo: Reliable Causal Chain Reasoning via Structural Causal Recurrent Neural Networks](https://mlsanity.com/paper/2212.08322/) (2022)
- [Synergizing Machine Learning & Symbolic Methods: A Survey on Hybrid Approaches to Natural Language Processing](https://mlsanity.com/paper/2401.11972/) (2024)
- [Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automata](https://mlsanity.com/paper/2404.06854/) (2024)
- [Hierarchical Narrative Analysis: Unraveling Perceptions of Generative AI](https://mlsanity.com/paper/2409.11032/) (2024)

## Abstract (arXiv)

We present a conditional text generation framework that posits sentential expressions of possible causes and effects. This framework depends on two novel resources we develop in the course of this work: a very large-scale collection of English sentences expressing causal patterns CausalBank; and a refinement over previous work on constructing large lexical causal knowledge graphs Cause Effect Graph. Further, we extend prior work in lexically-constrained decoding to support disjunctive positive constraints. Human assessment confirms that our approach gives high-quality and diverse outputs. Finally, we use CausalBank to perform continued training of an encoder supporting a recent state-of-the-art model for causal reasoning, leading to a 3-point improvement on the COPA challenge set, with no change in model architecture.