# Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction

arXiv: https://arxiv.org/abs/2308.02103 · HTML: https://mlsanity.com/paper/2308.02103/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 76 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [A Simple Method for Commonsense Reasoning](https://mlsanity.com/paper/1806.02847/) (2018)
- [WARP: Word-level Adversarial ReProgramming](https://mlsanity.com/paper/2101.00121/) (2021)
- [Commonsense Knowledge Mining from Pretrained Models](https://mlsanity.com/paper/1909.00505/) (2019)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification](https://mlsanity.com/paper/2010.13641/) (2020)
- [Word Representations via Gaussian Embedding](https://mlsanity.com/paper/1412.6623/) (2014)
- [ASER: A Large-scale Eventuality Knowledge Graph](https://mlsanity.com/paper/1905.00270/) (2019)
- [Counterfactual Story Reasoning and Generation](https://mlsanity.com/paper/1909.04076/) (2019)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [Quantifying Uncertainties in Natural Language Processing Tasks](https://mlsanity.com/paper/1811.07253/) (2018)
- [Prompt-Learning for Fine-Grained Entity Typing](https://mlsanity.com/paper/2108.10604/) (2021)
- [Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction](https://mlsanity.com/paper/2202.12109/) (2022)
- [Prototypical Verbalizer for Prompt-based Few-shot Tuning](https://mlsanity.com/paper/2203.09770/) (2022)
- [Dynamic Prefix-Tuning for Generative Template-based Event Extraction](https://mlsanity.com/paper/2205.06166/) (2022)
- [Constructing Narrative Event Evolutionary Graph for Script Event Prediction](https://mlsanity.com/paper/1805.05081/) (2018)

## Abstract (arXiv)

Script Event Prediction (SEP) aims to predict the subsequent event for a given event chain from a candidate list. Prior research has achieved great success by integrating external knowledge to enhance the semantics, but it is laborious to acquisite the appropriate knowledge resources and retrieve the script-related knowledge. In this paper, we regard public pre-trained language models as knowledge bases and automatically mine the script-related knowledge via prompt-learning. Still, the scenario-diversity and label-ambiguity in scripts make it uncertain to construct the most functional prompt and label token in prompt learning, i.e., prompt-uncertainty and verbalizer-uncertainty. Considering the innate ability of Gaussian distribution to express uncertainty, we deploy the prompt tokens and label tokens as random variables following Gaussian distributions, where a prompt estimator and a verbalizer estimator are proposed to estimate their probabilistic representations instead of deterministic representations. We take the lead to explore prompt-learning in SEP and provide a fresh perspective to enrich the script semantics. Our method is evaluated on the most widely used benchmark and a newly proposed large-scale one. Experiments show that our method, which benefits from knowledge evoked from pre-trained language models, outperforms prior baselines by 1.46% and 1.05% on two benchmarks, respectively.