mlsanityData to 7 Oct 2026

Procedural Knowledge Extraction from Industrial Troubleshooting Guides Using Vision Language Models

arXiv 2601.22754 ↗First posted 30 Jan 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Industrial troubleshooting guides encode diagnostic procedures in flowchart-like diagrams where spatial layout and technical language jointly convey meaning. To integrate this knowledge into operator support systems, which assist shop-floor personnel in diagnosing and resolving equipment issues, the information must first be extracted and structured for machine interpretation. However, when performed manually, this extraction is labor-intensive and error-prone. Vision Language Models offer potential to automate this process by jointly interpreting visual and textual meaning, yet their performance on such guides remains underexplored. This paper evaluates two VLMs on extracting structured knowledge, comparing two prompting strategies: standard instruction-guided versus an augmented approach that cues troubleshooting layout patterns. Results reveal model-specific trade-offs between layout sensitivity and semantic robustness, informing practical deployment decisions.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 15 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

9 of 15 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024FlowLearn: Evaluating Large Vision-Language Models on Flowchart Understanding6 Jul

Other references · 8

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  5. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  6. 2019Learning Household Task Knowledge from WikiHow Descriptionscited by 15
  7. 2022Reasoning about Procedures with Natural Language Processing: A Tutorialcited by 8
  8. 2024Human Evaluation of Procedural Knowledge Graph Extraction from Text with Large Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026FlowExtract: Procedural Knowledge Extraction from Maintenance Flowchartscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams
  2. 2026A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection
  3. 2026Adapting Vision-Language Models for Human-Readable XAI in Industrial Object Detection
  4. 2025From Drawings to Decisions: A Hybrid Vision-Language Framework for Parsing 2D Engineering Drawings into Structured Manufacturing Knowledge
  5. 2026Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space
  6. 2026GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models
  7. 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design
  8. 2025Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?