mlsanityData to 14 Oct 2026

Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension

arXiv 1805.06975 ↗First posted 17 May 2018cs.CLcited by 62

Not on the map. Why?

Abstract (arXiv)

We present a new dataset and models for comprehending paragraphs about processes (e.g., photosynthesis), an important genre of text describing a dynamic world. The new dataset, ProPara, is the first to contain natural (rather than machine-generated) text about a changing world along with a full annotation of entity states (location and existence) during those changes (81k datapoints). The end-task, tracking the location and existence of entities through the text, is challenging because the causal effects of actions are often implicit and need to be inferred. We find that previous models that have worked well on synthetic data achieve only mediocre performance on ProPara, and introduce two new neural models that exploit alternative mechanisms for state prediction, in particular using LSTM input encoding and span prediction. The new models improve accuracy by up to 19%. The dataset and models are available to the community at http://data.allenai.org/propara.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 18; 2023: 13; 2024: 10; 2025: 0; 2026: 7.

62 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  3. 2022Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehensioncited by 80
  4. 2021General-Purpose Question-Answering with Macawcited by 40
  5. 2021ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualitiescited by 28
  6. 2023StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understandingcited by 26
  7. 2024SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videoscited by 24
  8. 2023Causal Reasoning of Entities and Events in Procedural Textscited by 23
  9. 2021Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Eventscited by 22
  10. 2022Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processescited by 22
  11. 2024Making Large Language Models into World Models with Precondition and Effect Knowledgecited by 22
  12. 2021proScript: Partially Ordered Scripts Generation via Pre-trained Language Modelscited by 21
38 more
  1. 2021Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understandingcited by 19
  2. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  3. 2021Process-Level Representation of Scientific Protocols with Interactive Annotationcited by 16
  4. 2022"John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibilitycited by 15
  5. 2021From Discourse to Narrative: Knowledge Projection for Event Relation Extractioncited by 14
  6. 2024MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Datasetcited by 11
  7. 2024CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Planscited by 11
  8. 2022Coalescing Global and Local Information for Procedural Text Understandingcited by 10
  9. 2024PlanLLM: Video Procedure Planning with Refinable Large Language Modelscited by 10
  10. 2023Understand the Dynamic World: An End-to-End Knowledge Informed Framework for Open Domain Entity State Trackingcited by 9
  11. 2022Reasoning about Procedures with Natural Language Processing: A Tutorialcited by 8
  12. 2022Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flowscited by 8
  13. 2023OpenPI2.0: An Improved Dataset for Entity Tracking in Textscited by 8
  14. 2023Localizing Active Objects from Egocentric Vision with Symbolic World Knowledgecited by 8
  15. 2023CRoW: Benchmarking Commonsense Reasoning in Real-World Taskscited by 8
  16. 2024ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogiescited by 8
  17. 2021Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarkingcited by 7
  18. 2022CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarizationcited by 7
  19. 2022EvEntS ReaLM: Event Reasoning of Entity States via Language Modelscited by 7
  20. 2022Language Modeling with Latent Situationscited by 6
  21. 2022LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-trainingcited by 5
  22. 2022PASTA: A Dataset for Modeling Participant States in Narrativescited by 5
  23. 2021Relational Gating for "What If" Reasoningcited by 4
  24. 2021SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domaincited by 4
  25. 2022Learning Action Conditions from Instructional Manuals for Instruction Understandingcited by 4
  26. 2022The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources in Natural Language Understanding Systemscited by 4
  27. 2023The Role of Semantic Parsing in Understanding Procedural Textcited by 4
  28. 2021Latent Alignment of Procedural Concepts in Multimodal Recipescited by 2
  29. 2021Tracking entities in technical procedures -- a new dataset and baselinescited by 2
  30. 202250 Ways to Bake a Cookie: Mapping the Landscape of Procedural Textscited by 2
  31. 2023Transferring Procedural Knowledge across Commonsense Taskscited by 2
  32. 2023From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoningcited by 2
  33. 2024Consistent Joint Decision-Making with Heterogeneous Learning Modelscited by 2
  34. 2024Order-Based Pre-training Strategies for Procedural Text Understandingcited by 2
  35. 2026Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QAcited by 2
  36. 2021A Neural Edge-Editing Approach for Document-Level Relation Graph Extractioncited by 1
  37. 2022Procedural Text Understanding via Scene-Wise Evolutioncited by 1
  38. 2022Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decodingcited by 1

50 most cited of 62 All 62 (JSON)