# OpenPI2.0: An Improved Dataset for Entity Tracking in Texts

arXiv: https://arxiv.org/abs/2305.14603 · HTML: https://mlsanity.com/paper/2305.14603/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [Translating Natural Language to Planning Goals with Large-Language Models](https://mlsanity.com/paper/2302.05128/) (2023)
- [WIQA: A dataset for "What if..." reasoning over procedural text](https://mlsanity.com/paper/1909.04739/) (2019)
- [Reasoning about Goals, Steps, and Temporal Ordering with WikiHow](https://mlsanity.com/paper/2009.07690/) (2020)
- [Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension](https://mlsanity.com/paper/1805.06975/) (2018)
- [Simulating Action Dynamics with Neural Process Networks](https://mlsanity.com/paper/1711.05313/) (2017)
- [The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures](https://mlsanity.com/paper/1905.06939/) (2019)
- [Visual Goal-Step Inference using wikiHow](https://mlsanity.com/paper/2104.05845/) (2021)
- [A Dataset for Tracking Entities in Open Domain Procedural Text](https://mlsanity.com/paper/2011.08092/) (2020)
- [Simpler Context-Dependent Logical Forms via Model Projections](https://mlsanity.com/paper/1606.05378/) (2016)
- [Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text](https://mlsanity.com/paper/1909.04745/) (2019)
- [Goal-Oriented Script Construction](https://mlsanity.com/paper/2107.13189/) (2021)
- [Analogous Process Structure Induction for Sub-event Sequence Prediction](https://mlsanity.com/paper/2010.08525/) (2020)
- [Causal Reasoning of Entities and Events in Procedural Texts](https://mlsanity.com/paper/2301.10896/) (2023)
- [Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data](https://mlsanity.com/paper/2203.07264/) (2022)
- [Intent Detection with WikiHow](https://mlsanity.com/paper/2009.05781/) (2020)
- [Tracking Discrete and Continuous Entity State for Process Understanding](https://mlsanity.com/paper/1904.03518/) (2019)
- [Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video Retrieval](https://mlsanity.com/paper/2111.09276/) (2021)
- [Human-in-the-Loop Schema Induction](https://mlsanity.com/paper/2302.13048/) (2023)
- [Be Consistent! Improving Procedural Text Comprehension using Label Consistency](https://mlsanity.com/paper/1906.08942/) (2019)
- [What-if I ask you to explain: Explaining the effects of perturbations in procedural text](https://mlsanity.com/paper/2005.01526/) (2020)

## Built on by (most cited first, 8 of 8)

- [SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos](https://mlsanity.com/paper/2403.01599/) (2024)
- [On the Limit of Language Models as Planning Formalizers](https://mlsanity.com/paper/2412.09879/) (2024)
- [CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans](https://mlsanity.com/paper/2406.15823/) (2024)
- [Chain and Causal Attention for Efficient Entity Tracking](https://mlsanity.com/paper/2410.05565/) (2024)
- [EnigmaToM: Improve LLMs' Theory-of-Mind Reasoning Capabilities with Neural Knowledge Base of Entity States](https://mlsanity.com/paper/2503.03340/) (2025)
- [Structured Event Reasoning with Large Language Models](https://mlsanity.com/paper/2408.16098/) (2024)
- [EvolvTrip: Enhancing Literary Character Understanding with Temporal Theory-of-Mind Graphs](https://mlsanity.com/paper/2506.13641/) (2025)
- [Controllable Narrative Rendering for Enhanced Assisted Writing](https://mlsanity.com/paper/2607.00009/) (2026)

## Abstract (arXiv)

Much text describes a changing world (e.g., procedures, stories, newswires), and understanding them requires tracking how entities change. An earlier dataset, OpenPI, provided crowdsourced annotations of entity state changes in text. However, a major limitation was that those annotations were free-form and did not identify salient changes, hampering model evaluation. To overcome these limitations, we present an improved dataset, OpenPI2.0, where entities and attributes are fully canonicalized and additional entity salience annotations are added. On our fairer evaluation setting, we find that current state-of-the-art language models are far from competent. We also show that using state changes of salient entities as a chain-of-thought prompt, downstream performance is improved on tasks such as question answering and classical planning, outperforming the setting involving all related entities indiscriminately. We offer OpenPI2.0 for the continued development of models that can understand the dynamics of entities in text.