# Semantic Frame Forecast

arXiv: https://arxiv.org/abs/2104.05604 · HTML: https://mlsanity.com/paper/2104.05604/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 12 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016)
- [Strategies for Structuring Story Generation](https://mlsanity.com/paper/1902.01109/) (2019)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Do Massively Pretrained Language Models Make Better Storytellers?](https://mlsanity.com/paper/1909.10705/) (2019)
- [Event Representations for Automated Story Generation with Deep Neural Nets](https://mlsanity.com/paper/1706.01331/) (2017)
- [Narrative Planning: Balancing Plot and Character](https://mlsanity.com/paper/1401.3841/) (2014)
- [Story Realization: Expanding Plot Events into Sentences](https://mlsanity.com/paper/1909.03480/) (2019)
- [Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold](https://mlsanity.com/paper/1706.09528/) (2017)
- [Narrative Interpolation for Generating and Understanding Stories](https://mlsanity.com/paper/2008.07466/) (2020)
- [Progressive Generation of Long Text with Pretrained Language Models](https://mlsanity.com/paper/2006.15720/) (2020)
- [Heteroglossia: In-Situ Story Ideation with the Crowd](https://mlsanity.com/paper/2001.04519/) (2020)
- [On the Evaluation of Dialogue Systems with Next Utterance Classification](https://mlsanity.com/paper/1605.05414/) (2016)
- [Visual Storytelling via Predicting Anchor Word Embeddings in the Stories](https://mlsanity.com/paper/2001.04541/) (2020)

## Built on by (most cited first, 3 of 3)

- [Summaries as Captions: Generating Figure Captions for Scientific Documents with Automated Text Summarization](https://mlsanity.com/paper/2302.12324/) (2023)
- [Semantically-informed Hierarchical Event Modeling](https://mlsanity.com/paper/2212.10547/) (2022)
- [Conveying the Predicted Future to Users: A Case Study of Story Plot Prediction](https://mlsanity.com/paper/2302.09122/) (2023)

## Abstract (arXiv)

This paper introduces semantic frame forecast, a task that predicts the semantic frames that will occur in the next 10, 100, or even 1,000 sentences in a running story. Prior work focused on predicting the immediate future of a story, such as one to a few sentences ahead. However, when novelists write long stories, generating a few sentences is not enough to help them gain high-level insight to develop the follow-up story. In this paper, we formulate a long story as a sequence of "story blocks," where each block contains a fixed number of sentences (e.g., 10, 100, or 200). This formulation allows us to predict the follow-up story arc beyond the scope of a few sentences. We represent a story block using the term frequencies (TF) of semantic frames in it, normalized by each frame's inverse document frequency (IDF). We conduct semantic frame forecast experiments on 4,794 books from the Bookcorpus and 7,962 scientific abstracts from CODA-19, with block sizes ranging from 5 to 1,000 sentences. The results show that automated models can forecast the follow-up story blocks better than the random, prior, and replay baselines, indicating the task's feasibility. We also learn that the models using the frame representation as features outperform all the existing approaches when the block size is over 150 sentences. The human evaluation also shows that the proposed frame representation, when visualized as word clouds, is comprehensible, representative, and specific to humans. Our code is available at https://github.com/appleternity/FrameForecasting.