# Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension

arXiv: https://arxiv.org/abs/1805.06975 · HTML: https://mlsanity.com/paper/1805.06975/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2018.

## Built on by (most cited first, 30 of 62)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Language Models of Code are Few-Shot Commonsense Learners](https://mlsanity.com/paper/2210.07128/) (2022)
- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [General-Purpose Question-Answering with Macaw](https://mlsanity.com/paper/2109.02593/) (2021)
- [ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualities](https://mlsanity.com/paper/2104.02137/) (2021)
- [StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding](https://mlsanity.com/paper/2310.12874/) (2023)
- [SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos](https://mlsanity.com/paper/2403.01599/) (2024)
- [Causal Reasoning of Entities and Events in Procedural Texts](https://mlsanity.com/paper/2301.10896/) (2023)
- [Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events](https://mlsanity.com/paper/2104.07635/) (2021)
- [Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processes](https://mlsanity.com/paper/2210.12197/) (2022)
- [Making Large Language Models into World Models with Precondition and Effect Knowledge](https://mlsanity.com/paper/2409.12278/) (2024)
- [proScript: Partially Ordered Scripts Generation via Pre-trained Language Models](https://mlsanity.com/paper/2104.08251/) (2021)
- [Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding](https://mlsanity.com/paper/2109.04947/) (2021)
- [A Systematic Survey of Text Worlds as Embodied Natural Language Environments](https://mlsanity.com/paper/2107.04132/) (2021)
- [Process-Level Representation of Scientific Protocols with Interactive Annotation](https://mlsanity.com/paper/2101.10244/) (2021)
- ["John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibility](https://mlsanity.com/paper/2210.07471/) (2022)
- [From Discourse to Narrative: Knowledge Projection for Event Relation Extraction](https://mlsanity.com/paper/2106.08629/) (2021)
- [MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset](https://mlsanity.com/paper/2406.02106/) (2024)
- [CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans](https://mlsanity.com/paper/2406.15823/) (2024)
- [Coalescing Global and Local Information for Procedural Text Understanding](https://mlsanity.com/paper/2208.12848/) (2022)
- [PlanLLM: Video Procedure Planning with Refinable Large Language Models](https://mlsanity.com/paper/2412.19139/) (2024)
- [Understand the Dynamic World: An End-to-End Knowledge Informed Framework for Open Domain Entity State Tracking](https://mlsanity.com/paper/2304.13854/) (2023)
- [Reasoning about Procedures with Natural Language Processing: A Tutorial](https://mlsanity.com/paper/2205.07455/) (2022)
- [Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows](https://mlsanity.com/paper/2209.05840/) (2022)
- [OpenPI2.0: An Improved Dataset for Entity Tracking in Texts](https://mlsanity.com/paper/2305.14603/) (2023)
- [Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge](https://mlsanity.com/paper/2310.15066/) (2023)
- [CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks](https://mlsanity.com/paper/2310.15239/) (2023)
- [ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies](https://mlsanity.com/paper/2403.01139/) (2024)
- [Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking](https://mlsanity.com/paper/2112.00086/) (2021)
- [CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization](https://mlsanity.com/paper/2210.14190/) (2022)

## Abstract (arXiv)

We present a new dataset and models for comprehending paragraphs about processes (e.g., photosynthesis), an important genre of text describing a dynamic world. The new dataset, ProPara, is the first to contain natural (rather than machine-generated) text about a changing world along with a full annotation of entity states (location and existence) during those changes (81k datapoints). The end-task, tracking the location and existence of entities through the text, is challenging because the causal effects of actions are often implicit and need to be inferred. We find that previous models that have worked well on synthetic data achieve only mediocre performance on ProPara, and introduce two new neural models that exploit alternative mechanisms for state prediction, in particular using LSTM input encoding and span prediction. The new models improve accuracy by up to 19%. The dataset and models are available to the community at http://data.allenai.org/propara.