# Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data

arXiv: https://arxiv.org/abs/2203.07264 · HTML: https://mlsanity.com/paper/2203.07264/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 62 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis](https://mlsanity.com/paper/1903.02874/) (2019)
- [Unsupervised Learning from Narrated Instruction Videos](https://mlsanity.com/paper/1506.09215/) (2015)
- [Scalable Zero-shot Entity Linking with Dense Entity Retrieval](https://mlsanity.com/paper/1911.03814/) (2019)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [WIQA: A dataset for "What if..." reasoning over procedural text](https://mlsanity.com/paper/1909.04739/) (2019)
- [Reasoning about Goals, Steps, and Temporal Ordering with WikiHow](https://mlsanity.com/paper/2009.07690/) (2020)
- [Visual Goal-Step Inference using wikiHow](https://mlsanity.com/paper/2104.05845/) (2021)
- [Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text](https://mlsanity.com/paper/1909.04745/) (2019)
- [Goal-Oriented Script Construction](https://mlsanity.com/paper/2107.13189/) (2021)
- [Analogous Process Structure Induction for Sub-event Sequence Prediction](https://mlsanity.com/paper/2010.08525/) (2020)
- [Simple and Effective Paraphrastic Similarity from Parallel Translations](https://mlsanity.com/paper/1909.13872/) (2019)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [Intent Detection with WikiHow](https://mlsanity.com/paper/2009.05781/) (2020)
- [Learning Household Task Knowledge from WikiHow Descriptions](https://mlsanity.com/paper/1909.06414/) (2019)
- [Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video Retrieval](https://mlsanity.com/paper/2111.09276/) (2021)
- [What-if I ask you to explain: Explaining the effects of perturbations in procedural text](https://mlsanity.com/paper/2005.01526/) (2020)
- [Integrating Know-How into the Linked Data Cloud](https://mlsanity.com/paper/1604.04506/) (2016)

## Built on by (most cited first, 22 of 22)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2505.16421/) (2025)
- [Procedure-Aware Pretraining for Instructional Video Understanding](https://mlsanity.com/paper/2303.18230/) (2023)
- [Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale](https://mlsanity.com/paper/2409.15637/) (2024)
- [Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification](https://mlsanity.com/paper/2307.01972/) (2023)
- [DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning](https://mlsanity.com/paper/2406.09953/) (2024)
- [Multimedia Generative Script Learning for Task Planning](https://mlsanity.com/paper/2208.12306/) (2022)
- [TaskLAMA: Probing the Complex Task Understanding of Language Models](https://mlsanity.com/paper/2308.15299/) (2023)
- [Reasoning about Procedures with Natural Language Processing: A Tutorial](https://mlsanity.com/paper/2205.07455/) (2022)
- [OpenPI2.0: An Improved Dataset for Entity Tracking in Texts](https://mlsanity.com/paper/2305.14603/) (2023)
- [Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generation](https://mlsanity.com/paper/2305.10907/) (2023)
- [Generating Illustrated Instructions](https://mlsanity.com/paper/2312.04552/) (2023)
- [Rewriting the Script: Adapting Text Instructions for Voice Interaction](https://mlsanity.com/paper/2306.09992/) (2023)
- [Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkish](https://mlsanity.com/paper/2309.06698/) (2023)
- [Schema-Guided Event Graph Completion](https://mlsanity.com/paper/2206.02921/) (2022)
- [WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment](https://mlsanity.com/paper/2407.07778/) (2024)
- [PARADISE: Evaluating Implicit Planning Skills of Language Models with Procedural Warnings and Tips Dataset](https://mlsanity.com/paper/2403.03167/) (2024)
- [Universal Retrieval for Multimodal Trajectory Modeling](https://mlsanity.com/paper/2506.22056/) (2025)
- [Large Language Models as Sous Chefs: Revising Recipes with GPT-3](https://mlsanity.com/paper/2306.13986/) (2023)
- [Choice-75: A Dataset on Decision Branching in Script Learning](https://mlsanity.com/paper/2309.11737/) (2023)
- [Towards Flow Graph Prediction of Open-Domain Procedural Texts](https://mlsanity.com/paper/2305.19497/) (2023)
- [PSSD: Making Large Language Models Self-denial via Human Psyche Structure](https://mlsanity.com/paper/2502.01344/) (2025)

## Abstract (arXiv)

Procedures are inherently hierarchical. To "make videos", one may need to "purchase a camera", which in turn may require one to "set a budget". While such hierarchical knowledge is critical for reasoning about complex procedures, most existing work has treated procedures as shallow structures without modeling the parent-child relation. In this work, we attempt to construct an open-domain hierarchical knowledge-base (KB) of procedures based on wikiHow, a website containing more than 110k instructional articles, each documenting the steps to carry out a complex procedure. To this end, we develop a simple and efficient method that links steps (e.g., "purchase a camera") in an article to other articles with similar goals (e.g., "how to choose a camera"), recursively constructing the KB. Our method significantly outperforms several strong baselines according to automatic evaluation, human judgment, and application to downstream tasks such as instructional video retrieval. A demo with partial data can be found at https://wikihow-hierarchy.github.io. The code and the data are at https://github.com/shuyanzhou/wikihow_hierarchy.