Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data
Not on the map. Why?
Abstract (arXiv)
Procedures are inherently hierarchical. To "make videos", one may need to "purchase a camera", which in turn may require one to "set a budget". While such hierarchical knowledge is critical for reasoning about complex procedures, most existing work has treated procedures as shallow structures without modeling the parent-child relation. In this work, we attempt to construct an open-domain hierarchical knowledge-base (KB) of procedures based on wikiHow, a website containing more than 110k instructional articles, each documenting the steps to carry out a complex procedure. To this end, we develop a simple and efficient method that links steps (e.g., "purchase a camera") in an article to other articles with similar goals (e.g., "how to choose a camera"), recursively constructing the KB. Our method significantly outperforms several strong baselines according to automatic evaluation, human judgment, and application to downstream tasks such as instructional video retrieval. A demo with partial data can be found at https://wikihow-hierarchy.github.io. The code and the data are at https://github.com/shuyanzhou/wikihow_hierarchy.
Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 28 of its 62 references that are in the map. First posted on arXiv 14 Mar 2022.
Builds on
28 of 62 references.
References · 28
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
- 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
- 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
16 more
- 2015Unsupervised Learning from Narrated Instruction Videoscited by 116
- 2019Scalable Zero-shot Entity Linking with Dense Entity Retrievalcited by 85
- 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
- 2019WIQA: A dataset for "What if..." reasoning over procedural textcited by 71
- 2020Reasoning about Goals, Steps, and Temporal Ordering with WikiHowcited by 66
- 2021Visual Goal-Step Inference using wikiHowcited by 42
- 2019Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Textcited by 31
- 2021Goal-Oriented Script Constructioncited by 30
- 2020Analogous Process Structure Induction for Sub-event Sequence Predictioncited by 26
- 2019Simple and Effective Paraphrastic Similarity from Parallel Translationscited by 25
- 2020Intent Detection with WikiHowcited by 19
- 2021Paraphrastic Representations at Scalecited by 19
- 2019Learning Household Task Knowledge from WikiHow Descriptionscited by 15
- 2021Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video Retrievalcited by 13
- 2020What-if I ask you to explain: Explaining the effects of perturbations in procedural textcited by 9
- 2016Integrating Know-How into the Linked Data Cloudcited by 4
Built on by
Citing papers per year: 2022: 3; 2023: 12; 2024: 4; 2025: 3.
22 papers.
Sort:
By area:
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
- 2023Procedure-Aware Pretraining for Instructional Video Understandingcited by 48
- 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verificationcited by 19
- 2024DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planningcited by 14
- 2022Multimedia Generative Script Learning for Task Planningcited by 12
- 2023TaskLAMA: Probing the Complex Task Understanding of Language Modelscited by 12
- 2022Reasoning about Procedures with Natural Language Processing: A Tutorialcited by 8
- 2023OpenPI2.0: An Improved Dataset for Entity Tracking in Textscited by 8
- 2023Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generationcited by 7
- 2023Generating Illustrated Instructionscited by 7
10 more
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interactioncited by 5
- 2023Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkishcited by 5
- 2022Schema-Guided Event Graph Completioncited by 4
- 2024WorldAPIs: The World Is Worth How Many APIs? A Thought Experimentcited by 4
- 2024PARADISE: Evaluating Implicit Planning Skills of Language Models with Procedural Warnings and Tips Datasetcited by 3
- 2025Universal Retrieval for Multimodal Trajectory Modelingcited by 3
- 2023Large Language Models as Sous Chefs: Revising Recipes with GPT-3cited by 2
- 2023Choice-75: A Dataset on Decision Branching in Script Learningcited by 2
- 2023Towards Flow Graph Prediction of Open-Domain Procedural Textscited by 0
- 2025PSSD: Making Large Language Models Self-denial via Human Psyche Structurecited by 0
- 2025Universal Retrieval for Multimodal Trajectory Modeling27 Jun 2025
- 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning22 May 2025
- 2025PSSD: Making Large Language Models Self-denial via Human Psyche Structure3 Feb 2025
- 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale24 Sep 2024
- 2024WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment10 Jul 2024
- 2024DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning14 Jun 2024
- 2024PARADISE: Evaluating Implicit Planning Skills of Language Models with Procedural Warnings and Tips Dataset5 Mar 2024
- 2023Generating Illustrated Instructions7 Dec 2023
- 2023Choice-75: A Dataset on Decision Branching in Script Learning21 Sep 2023
- 2023Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkish13 Sep 2023
- 2023TaskLAMA: Probing the Complex Task Understanding of Language Models29 Aug 2023
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul 2023
10 more
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification5 Jul 2023
- 2023Large Language Models as Sous Chefs: Revising Recipes with GPT-324 Jun 2023
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interaction16 Jun 2023
- 2023Towards Flow Graph Prediction of Open-Domain Procedural Texts31 May 2023
- 2023OpenPI2.0: An Improved Dataset for Entity Tracking in Texts24 May 2023
- 2023Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generation18 May 2023
- 2023Procedure-Aware Pretraining for Instructional Video Understanding31 Mar 2023
- 2022Multimedia Generative Script Learning for Task Planning25 Aug 2022
- 2022Schema-Guided Event Graph Completion6 Jun 2022
- 2022Reasoning about Procedures with Natural Language Processing: A Tutorial16 May 2022
- 2023Procedure-Aware Pretraining for Instructional Video Understandingcited by 48
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verificationcited by 19
- 2024DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planningcited by 14
- 2022Multimedia Generative Script Learning for Task Planningcited by 12
- 2023TaskLAMA: Probing the Complex Task Understanding of Language Modelscited by 12
- 2022Reasoning about Procedures with Natural Language Processing: A Tutorialcited by 8
- 2023OpenPI2.0: An Improved Dataset for Entity Tracking in Textscited by 8
- 2023Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generationcited by 7
- 2023Generating Illustrated Instructionscited by 7
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interactioncited by 5
- 2023Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkishcited by 5
- 2022Schema-Guided Event Graph Completioncited by 4
12 most cited of 17 All 17 (JSON)
- 2025Universal Retrieval for Multimodal Trajectory Modeling27 Jun 2025
- 2024DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning14 Jun 2024
- 2024PARADISE: Evaluating Implicit Planning Skills of Language Models with Procedural Warnings and Tips Dataset5 Mar 2024
- 2023Generating Illustrated Instructions7 Dec 2023
- 2023Choice-75: A Dataset on Decision Branching in Script Learning21 Sep 2023
- 2023Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkish13 Sep 2023
- 2023TaskLAMA: Probing the Complex Task Understanding of Language Models29 Aug 2023
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification5 Jul 2023
- 2023Large Language Models as Sous Chefs: Revising Recipes with GPT-324 Jun 2023
- 2023Rewriting the Script: Adapting Text Instructions for Voice Interaction16 Jun 2023
- 2023Towards Flow Graph Prediction of Open-Domain Procedural Texts31 May 2023
- 2023OpenPI2.0: An Improved Dataset for Entity Tracking in Texts24 May 2023
12 newest of 17 All 17 (JSON)