# Choice-75: A Dataset on Decision Branching in Script Learning

arXiv: https://arxiv.org/abs/2309.11737 · HTML: https://mlsanity.com/paper/2309.11737/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://mlsanity.com/paper/2201.05955/) (2022)
- [Reasoning about Goals, Steps, and Temporal Ordering with WikiHow](https://mlsanity.com/paper/2009.07690/) (2020)
- [Generating Counter Narratives against Online Hate Speech: Data and Strategies](https://mlsanity.com/paper/2004.04216/) (2020)
- [Temporal Reasoning on Implicit Events from Distant Supervision](https://mlsanity.com/paper/2010.12753/) (2020)
- [A Dataset for Tracking Entities in Open Domain Procedural Text](https://mlsanity.com/paper/2011.08092/) (2020)
- [Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text](https://mlsanity.com/paper/1909.04745/) (2019)
- [Goal-Oriented Script Construction](https://mlsanity.com/paper/2107.13189/) (2021)
- [Causal Reasoning of Entities and Events in Procedural Texts](https://mlsanity.com/paper/2301.10896/) (2023)
- [Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data](https://mlsanity.com/paper/2203.07264/) (2022)
- [COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspective](https://mlsanity.com/paper/2305.05191/) (2023)
- [SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets](https://mlsanity.com/paper/2111.06467/) (2021)
- [Modeling financial analysts' decision making via the pragmatics and semantics of earnings calls](https://mlsanity.com/paper/1906.02868/) (2019)
- [Towards Socially Intelligent Agents with Mental State Transition and Human Utility](https://mlsanity.com/paper/2103.07011/) (2021)
- [Human-in-the-Loop Schema Induction](https://mlsanity.com/paper/2302.13048/) (2023)
- [Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1910.09260/) (2019)
- [What makes you change your mind? An empirical investigation in online group decision-making conversations](https://mlsanity.com/paper/2207.12035/) (2022)

## Built on by (most cited first, 2 of 2)

- [CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans](https://mlsanity.com/paper/2406.15823/) (2024)
- [Tailoring with Targeted Precision: Edit-Based Agents for Open-Domain Procedure Customization](https://mlsanity.com/paper/2311.09510/) (2023)

## Abstract (arXiv)

Script learning studies how stereotypical events unfold, enabling machines to reason about narratives with implicit information. Previous works mostly consider a script as a linear sequence of events while ignoring the potential branches that arise due to people's circumstantial choices. We hence propose Choice-75, the first benchmark that challenges intelligent systems to make decisions given descriptive scenarios, containing 75 scripts and more than 600 scenarios. We also present preliminary results with current large language models (LLM). Although they demonstrate overall decent performance, there is still notable headroom in hard scenarios.