# Simulating Action Dynamics with Neural Process Networks

arXiv: https://arxiv.org/abs/1711.05313 · HTML: https://mlsanity.com/paper/1711.05313/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2017.

## Built on by (most cited first, 30 of 53)

- [Entity Tracking in Language Models](https://mlsanity.com/paper/2305.02363/) (2023)
- [Causal Reasoning of Entities and Events in Procedural Texts](https://mlsanity.com/paper/2301.10896/) (2023)
- [Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events](https://mlsanity.com/paper/2104.07635/) (2021)
- [Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processes](https://mlsanity.com/paper/2210.12197/) (2022)
- [proScript: Partially Ordered Scripts Generation via Pre-trained Language Models](https://mlsanity.com/paper/2104.08251/) (2021)
- [Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding](https://mlsanity.com/paper/2109.04947/) (2021)
- [Process-Level Representation of Scientific Protocols with Interactive Annotation](https://mlsanity.com/paper/2101.10244/) (2021)
- [SHARE: a System for Hierarchical Assistive Recipe Editing](https://mlsanity.com/paper/2105.08185/) (2021)
- [CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans](https://mlsanity.com/paper/2406.15823/) (2024)
- [Coalescing Global and Local Information for Procedural Text Understanding](https://mlsanity.com/paper/2208.12848/) (2022)
- [Understand the Dynamic World: An End-to-End Knowledge Informed Framework for Open Domain Entity State Tracking](https://mlsanity.com/paper/2304.13854/) (2023)
- [Text2TimeSeries: Enhancing Financial Forecasting through Time Series Prediction Updates with Event-Driven Insights from Large Language Models](https://mlsanity.com/paper/2407.03689/) (2024)
- [Reasoning about Procedures with Natural Language Processing: A Tutorial](https://mlsanity.com/paper/2205.07455/) (2022)
- [Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows](https://mlsanity.com/paper/2209.05840/) (2022)
- [OpenPI2.0: An Improved Dataset for Entity Tracking in Texts](https://mlsanity.com/paper/2305.14603/) (2023)
- [Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge](https://mlsanity.com/paper/2310.15066/) (2023)
- [EvEntS ReaLM: Event Reasoning of Entity States via Language Models](https://mlsanity.com/paper/2211.05392/) (2022)
- [Hallmarks of Human-Machine Collaboration: A framework for assessment in the DARPA Communicating with Computers Program](https://mlsanity.com/paper/2102.04958/) (2021)
- [LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training](https://mlsanity.com/paper/2201.08081/) (2022)
- [PASTA: A Dataset for Modeling Participant States in Narratives](https://mlsanity.com/paper/2208.00329/) (2022)
- [Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenario](https://mlsanity.com/paper/2210.11431/) (2022)
- [Plug-and-Play Recipe Generation with Content Planning](https://mlsanity.com/paper/2212.05093/) (2022)
- [Tailoring with Targeted Precision: Edit-Based Agents for Open-Domain Procedure Customization](https://mlsanity.com/paper/2311.09510/) (2023)
- [SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain](https://mlsanity.com/paper/2106.01972/) (2021)
- [Learning Action Conditions from Instructional Manuals for Instruction Understanding](https://mlsanity.com/paper/2205.12420/) (2022)
- [The Role of Semantic Parsing in Understanding Procedural Text](https://mlsanity.com/paper/2302.06829/) (2023)
- [MLD-EA: Check and Complete Narrative Coherence by Introducing Emotions and Actions](https://mlsanity.com/paper/2412.02897/) (2024)
- [Latent Alignment of Procedural Concepts in Multimodal Recipes](https://mlsanity.com/paper/2101.04727/) (2021)
- [Tracking entities in technical procedures -- a new dataset and baselines](https://mlsanity.com/paper/2104.07378/) (2021)
- [Modeling Temporal-Modal Entity Graph for Procedural Multimodal Machine Comprehension](https://mlsanity.com/paper/2204.02566/) (2022)

## Abstract (arXiv)

Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated. In this work, we introduce Neural Process Networks to understand procedural text through (neural) simulation of action dynamics. Our model complements existing memory architectures with dynamic entity tracking by explicitly modeling actions as state transformers. The model updates the states of the entities by executing learned action operators. Empirical results demonstrate that our proposed model can reason about the unstated causal effects of actions, allowing it to provide more accurate contextual information for understanding and generating procedural text, all while offering more interpretable internal representations than existing alternatives.