# Constructing Flow Graphs from Procedural Cybersecurity Texts

arXiv: https://arxiv.org/abs/2105.14357 · HTML: https://mlsanity.com/paper/2105.14357/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 54 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Graph-Bert: Only Attention is Needed for Learning Graph Representations](https://mlsanity.com/paper/2001.05140/) (2020)
- [Pre-training of Graph Augmented Transformers for Medication Recommendation](https://mlsanity.com/paper/1906.00346/) (2019)
- [Dynamically Fused Graph Network for Multi-hop Reasoning](https://mlsanity.com/paper/1905.06933/) (2019)
- [The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures](https://mlsanity.com/paper/1905.06939/) (2019)
- [VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification](https://mlsanity.com/paper/2004.05707/) (2020)
- [A Context-Aware Citation Recommendation Model with BERT and Graph Convolutional Networks](https://mlsanity.com/paper/1903.06464/) (2019)
- [Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension](https://mlsanity.com/paper/2005.05806/) (2020)
- [An Approach for Process Model Extraction By Multi-Grained Text Classification](https://mlsanity.com/paper/1906.02127/) (2019)
- [Multi-modal Cooking Workflow Construction for Food Recipes](https://mlsanity.com/paper/2008.09151/) (2020)
- [Graph Star Net for Generalized Multi-Task Learning](https://mlsanity.com/paper/1906.12330/) (2019)
- [A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos](https://mlsanity.com/paper/2005.00706/) (2020)
- [Cybersecurity Knowledge and Skills Taught in Capture the Flag Challenges](https://mlsanity.com/paper/2101.01421/) (2021)
- [Creative Procedural-Knowledge Extraction From Web Design Tutorials](https://mlsanity.com/paper/1904.08587/) (2019)

## Built on by (most cited first, 8 of 8)

- [Ontology-Aware RAG for Improved Question-Answering in Cybersecurity Education](https://mlsanity.com/paper/2412.14191/) (2024)
- [Unsupervised Task Graph Generation from Instructional Video Transcripts](https://mlsanity.com/paper/2302.09173/) (2023)
- [Learning Action Conditions from Instructional Manuals for Instruction Understanding](https://mlsanity.com/paper/2205.12420/) (2022)
- [The Role of Semantic Parsing in Understanding Procedural Text](https://mlsanity.com/paper/2302.06829/) (2023)
- [PAGED: A Benchmark for Procedural Graphs Extraction from Documents](https://mlsanity.com/paper/2408.03630/) (2024)
- [Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain](https://mlsanity.com/paper/2302.10346/) (2023)
- [Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs](https://mlsanity.com/paper/2505.18542/) (2025)
- [Multi-Agent Procedural Graph Extraction with Structural and Logical Refinement](https://mlsanity.com/paper/2601.19170/) (2026)

## Abstract (arXiv)

Following procedural texts written in natural languages is challenging. We must read the whole text to identify the relevant information or identify the instruction flows to complete a task, which is prone to failures. If such texts are structured, we can readily visualize instruction-flows, reason or infer a particular step, or even build automated systems to help novice agents achieve a goal. However, this structure recovery task is a challenge because of such texts' diverse nature. This paper proposes to identify relevant information from such texts and generate information flows between sentences. We built a large annotated procedural text dataset (CTFW) in the cybersecurity domain (3154 documents). This dataset contains valuable instructions regarding software vulnerability analysis experiences. We performed extensive experiments on CTFW with our LM-GNN model variants in multiple settings. To show the generalizability of both this task and our method, we also experimented with procedural texts from two other domains (Maintenance Manual and Cooking), which are substantially different from cybersecurity. Our experiments show that Graph Convolution Network with BERT sentence embeddings outperforms BERT in all three domains