# Learning Action Conditions from Instructional Manuals for Instruction Understanding

arXiv: https://arxiv.org/abs/2205.12420 · HTML: https://mlsanity.com/paper/2205.12420/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 52 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [Self-training Improves Pre-training for Natural Language Understanding](https://mlsanity.com/paper/2010.02194/) (2020)
- [A Multi-Axis Annotation Scheme for Event Temporal Relations](https://mlsanity.com/paper/1804.07828/) (2018)
- [WIQA: A dataset for "What if..." reasoning over procedural text](https://mlsanity.com/paper/1909.04739/) (2019)
- [Reasoning about Goals, Steps, and Temporal Ordering with WikiHow](https://mlsanity.com/paper/2009.07690/) (2020)
- [Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension](https://mlsanity.com/paper/1805.06975/) (2018)
- [Temporal Reasoning on Implicit Events from Distant Supervision](https://mlsanity.com/paper/2010.12753/) (2020)
- [Simulating Action Dynamics with Neural Process Networks](https://mlsanity.com/paper/1711.05313/) (2017)
- [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019)
- [Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages](https://mlsanity.com/paper/2010.03179/) (2020)
- [PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World](https://mlsanity.com/paper/2106.00188/) (2021)
- [Temporal Common Sense Acquisition with Minimal Supervision](https://mlsanity.com/paper/2005.04304/) (2020)
- [ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning](https://mlsanity.com/paper/2012.15283/) (2020)
- [A Dataset for Tracking Entities in Open Domain Procedural Text](https://mlsanity.com/paper/2011.08092/) (2020)
- [Document-level Entity-based Extraction as Template Generation](https://mlsanity.com/paper/2109.04901/) (2021)
- [Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text](https://mlsanity.com/paper/1909.04745/) (2019)
- [Reasoning about Actions and State Changes by Injecting Commonsense Knowledge](https://mlsanity.com/paper/1808.10012/) (2018)
- [Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals](https://mlsanity.com/paper/2110.08486/) (2021)
- [EventPlus: A Temporal Event Understanding Pipeline](https://mlsanity.com/paper/2101.04922/) (2021)
- [Learning Household Task Knowledge from WikiHow Descriptions](https://mlsanity.com/paper/1909.06414/) (2019)
- [Be Consistent! Improving Procedural Text Comprehension using Label Consistency](https://mlsanity.com/paper/1906.08942/) (2019)
- [Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging](https://mlsanity.com/paper/1808.09733/) (2018)
- [Modeling Preconditions in Text with a Crowd-sourced Dataset](https://mlsanity.com/paper/2010.02429/) (2020)

## Built on by (most cited first, 4 of 4)

- [Making Large Language Models into World Models with Precondition and Effect Knowledge](https://mlsanity.com/paper/2409.12278/) (2024)
- [Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge](https://mlsanity.com/paper/2310.15066/) (2023)
- [Inductive Bias Extraction and Matching for LLM Prompts](https://mlsanity.com/paper/2508.10295/) (2025)
- [Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models](https://mlsanity.com/paper/2606.03624/) (2026)

## Abstract (arXiv)

The ability to infer pre- and postconditions of an action is vital for comprehending complex instructions, and is essential for applications such as autonomous instruction-guided agents and assistive AI that supports humans to perform physical tasks. In this work, we propose a task dubbed action condition inference, and collecting a high-quality, human annotated dataset of preconditions and postconditions of actions in instructional manuals. We propose a weakly supervised approach to automatically construct large-scale training instances from online instructional manuals, and curate a densely human-annotated and validated dataset to study how well the current NLP models can infer action-condition dependencies in the instruction texts. We design two types of models differ by whether contextualized and global information is leveraged, as well as various combinations of heuristics to construct the weak supervisions. Our experimental results show a >20% F1-score improvement with considering the entire instruction contexts and a >6% F1-score benefit with the proposed heuristics.