# A Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event Extraction

arXiv: https://arxiv.org/abs/2305.15051 · HTML: https://mlsanity.com/paper/2305.15051/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 101 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Zero-Shot Transfer Learning for Event Extraction](https://mlsanity.com/paper/1707.01066/) (2017)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction](https://mlsanity.com/paper/2010.03009/) (2020)
- [One for All: Neural Joint Modeling of Entities and Events](https://mlsanity.com/paper/1812.00195/) (2018)
- [DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering](https://mlsanity.com/paper/2005.00697/) (2020)
- [Unsupervised Label-aware Event Trigger and Argument Classification](https://mlsanity.com/paper/2012.15243/) (2020)
- [Reading the Manual: Event Extraction as Definition Comprehension](https://mlsanity.com/paper/1912.01586/) (2019)
- [A Survey on Deep Learning Event Extraction: Approaches and Applications](https://mlsanity.com/paper/2107.02126/) (2021)
- [Efficient Zero-shot Event Extraction with Context-Definition Alignment](https://mlsanity.com/paper/2211.05156/) (2022)
- [Automatic Detection of Influential Actors in Disinformation Networks](https://mlsanity.com/paper/2005.10879/) (2020)
- [An Ordinal Latent Variable Model of Conflict Intensity](https://mlsanity.com/paper/2210.03971/) (2022)
- [Classifying Dyads for Militarized Conflict Analysis](https://mlsanity.com/paper/2109.12860/) (2021)
- [Improving Zero-Shot Event Extraction via Sentence Simplification](https://mlsanity.com/paper/2204.02531/) (2022)
- [EventGraph: Event Extraction as Semantic Graph Parsing](https://mlsanity.com/paper/2210.08646/) (2022)
- [1Cademy @ Causal News Corpus 2022: Leveraging Self-Training in Causality Classification of Socio-Political Event Data](https://mlsanity.com/paper/2211.02729/) (2022)
- [Generating Politically-Relevant Event Data](https://mlsanity.com/paper/1609.06239/) (2016)

## Built on by (most cited first, 2 of 2)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification](https://mlsanity.com/paper/2308.07876/) (2023)

## Abstract (arXiv)

Current social science efforts automatically populate event databases of "who did what to whom?" tuples, by applying event extraction (EE) to text such as news. The event databases are used to analyze sociopolitical dynamics between actor pairs (dyads) in, e.g., international relations. While most EE methods heavily rely on rules or supervised learning, zero-shot event extraction could potentially allow researchers to flexibly specify arbitrary event classes for new research questions. Unfortunately, we find that current zero-shot EE methods, as well as a naive zero-shot approach of simple generative language model (LM) prompting, perform poorly for dyadic event extraction; most suffer from word sense ambiguity, modality sensitivity, and computational inefficiency. We address these challenges with a new fine-grained, multi-stage instruction-following generative LM pipeline, proposing a Monte Carlo approach to deal with, and even take advantage of, nondeterminism of generative outputs. Our pipeline includes explicit stages of linguistic analysis (synonym generation, contextual disambiguation, argument realization, event modality), textitimproving control and interpretability compared to purely neural methods. This method outperforms other zero-shot EE approaches, and outperforms naive applications of generative LMs by at least 17 F1 percent points. The pipeline's filtering mechanism greatly improves computational efficiency, allowing it to perform as few as 12% of queries that a previous zero-shot method uses. Finally, we demonstrate our pipeline's application to dyadic international relations analysis.