# What Makes the Story Forward? Inferring Commonsense Explanations as Prompts for Future Event Generation

arXiv: https://arxiv.org/abs/2201.07099 · HTML: https://mlsanity.com/paper/2201.07099/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 18 Jan 2022.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Abductive Commonsense Reasoning](https://mlsanity.com/paper/1908.05739/) (2019)
- [COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs](https://mlsanity.com/paper/2010.05953/) (2020)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [Unsupervised Commonsense Question Answering with Self-Talk](https://mlsanity.com/paper/2004.05483/) (2020)
- [Strategies for Structuring Story Generation](https://mlsanity.com/paper/1902.01109/) (2019)
- [A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation](https://mlsanity.com/paper/2001.05139/) (2020)
- [Content Planning for Neural Story Generation with Aristotelian Rescoring](https://mlsanity.com/paper/2009.09870/) (2020)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models](https://mlsanity.com/paper/2010.00840/) (2020)
- [Reasoning about Goals, Steps, and Temporal Ordering with WikiHow](https://mlsanity.com/paper/2009.07690/) (2020)
- [Event Representations for Automated Story Generation with Deep Neural Nets](https://mlsanity.com/paper/1706.01331/) (2017)
- [Long and Diverse Text Generation with Planning-based Hierarchical Variational Model](https://mlsanity.com/paper/1908.06605/) (2019)
- [Modeling Naive Psychology of Characters in Simple Commonsense Stories](https://mlsanity.com/paper/1805.06533/) (2018)
- [Story Ending Generation with Incremental Encoding and Commonsense Knowledge](https://mlsanity.com/paper/1808.10113/) (2018)

## Built on by (most cited first, 8 of 8)

- [Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2212.04634/) (2022)
- [Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering](https://mlsanity.com/paper/2410.04752/) (2024)
- [Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer](https://mlsanity.com/paper/2305.12761/) (2023)
- [Probing Script Knowledge from Pre-Trained Models](https://mlsanity.com/paper/2204.10176/) (2022)
- [Unveiling Divergent Inductive Biases of LLMs on Temporal Data](https://mlsanity.com/paper/2404.01453/) (2024)
- [Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM](https://mlsanity.com/paper/2409.09362/) (2024)
- [Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models](https://mlsanity.com/paper/2601.00003/) (2025)
- [R2F: A General Retrieval, Reading and Fusion Framework for Document-level Natural Language Inference](https://mlsanity.com/paper/2210.12328/) (2022)

## Abstract (arXiv)

Prediction over event sequences is critical for many real-world applications in Information Retrieval and Natural Language Processing. Future Event Generation (FEG) is a challenging task in event sequence prediction because it requires not only fluent text generation but also commonsense reasoning to maintain the logical coherence of the entire event story. In this paper, we propose a novel explainable FEG framework, Coep. It highlights and integrates two types of event knowledge, sequential knowledge of direct event-event relations and inferential knowledge that reflects the intermediate character psychology between events, such as intents, causes, reactions, which intrinsically pushes the story forward. To alleviate the knowledge forgetting issue, we design two modules, Im and Gm, for each type of knowledge, which are combined via prompt tuning. First, Im focuses on understanding inferential knowledge to generate commonsense explanations and provide a soft prompt vector for Gm. We also design a contrastive discriminator for better generalization ability. Second, Gm generates future events by modeling direct sequential knowledge with the guidance of Im. Automatic and human evaluation demonstrate that our approach can generate more coherent, specific, and logical future events.