# From Engineering Diagrams to Graphs: Digitizing P&IDs with Transformers

arXiv: https://arxiv.org/abs/2411.13929 · HTML: https://mlsanity.com/paper/2411.13929/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 50 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 14 of 14)

- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [Benchmarking Detection Transfer Learning with Vision Transformers](https://mlsanity.com/paper/2111.11429/) (2021)
- [SGTR: End-to-end Scene Graph Generation with Transformer](https://mlsanity.com/paper/2112.12970/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [Relationformer: A Unified Framework for Image-to-Graph Generation](https://mlsanity.com/paper/2203.10202/) (2022)
- [Sat2Graph: Road Graph Extraction through Graph-Tensor Encoding](https://mlsanity.com/paper/2007.09547/) (2020)
- [Image-Conditioned Graph Generation for Road Network Extraction](https://mlsanity.com/paper/1910.14388/) (2019)
- [Automatic Information Extraction from Piping and Instrumentation Diagrams](https://mlsanity.com/paper/1901.11383/) (2019)
- [Rethinking the Evaluation of Unbiased Scene Graph Generation](https://mlsanity.com/paper/2208.01909/) (2022)
- [Digitize-PID: Automatic Digitization of Piping and Instrumentation Diagrams](https://mlsanity.com/paper/2109.03794/) (2021)
- [OSSR-PID: One-Shot Symbol Recognition in P&ID Sheets using Path Sampling and GCN](https://mlsanity.com/paper/2109.03849/) (2021)
- [Instance Segmentation Based Graph Extraction for Handwritten Circuit Diagram Images](https://mlsanity.com/paper/2301.03155/) (2023)

## Built on by (most cited first, 5 of 5)

- [Enginuity: Building an Open Multi-Domain Dataset of Complex Engineering Diagrams](https://mlsanity.com/paper/2601.13299/) (2026)
- [Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies](https://mlsanity.com/paper/2607.29389/) (2026)
- [A document is worth a structured record: Principled inductive bias design for document recognition](https://mlsanity.com/paper/2507.08458/) (2025)
- [FlowExtract: Procedural Knowledge Extraction from Maintenance Flowcharts](https://mlsanity.com/paper/2604.06770/) (2026)
- [SynthPID: P&ID digitization from Topology-Preserving Synthetic Data](https://mlsanity.com/paper/2604.16513/) (2026)

## Abstract (arXiv)

Digitizing engineering diagrams like Piping and Instrumentation Diagrams (P&IDs) plays a vital role in maintainability and operational efficiency of process and hydraulic systems. Previous methods typically decompose the task into separate steps such as symbol detection and line detection, which can limit their ability to capture the structure in these diagrams. In this work, a transformer-based approach leveraging the Relationformer that addresses this limitation by jointly extracting symbols and their interconnections from P&IDs is introduced. To evaluate our approach and compare it to a modular digitization approach, we present the first publicly accessible benchmark dataset for P&ID digitization, annotated with graph-level ground truth. Experimental results on real-world diagrams show that our method significantly outperforms the modular baseline, achieving over 25% improvement in edge detection accuracy. This research contributes a reproducible evaluation framework and demonstrates the effectiveness of transformer models for structural understanding of complex engineering diagrams. The dataset is available under https://zenodo.org/records/14803338.