# Learning to Ground Before Reading: Unified PCB Engineering Drawing Parsing with Compact Vision-Language Models

arXiv: https://arxiv.org/abs/2608.29268 · HTML: https://mlsanity.com/paper/2608.29268/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 29 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 20 of 20)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis](https://mlsanity.com/paper/2206.01062/) (2022)
- [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://mlsanity.com/paper/2110.00061/) (2021)
- [MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing](https://mlsanity.com/paper/2509.22186/) (2025)
- [LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis](https://mlsanity.com/paper/2103.15348/) (2021)
- [From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design](https://mlsanity.com/paper/2311.12668/) (2023)
- [Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction](https://mlsanity.com/paper/2411.03707/) (2024)
- [UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition](https://mlsanity.com/paper/2409.13148/) (2024)
- [From Drawings to Decisions: A Hybrid Vision-Language Framework for Parsing 2D Engineering Drawings into Structured Manufacturing Knowledge](https://mlsanity.com/paper/2506.17374/) (2025)

## Abstract (arXiv)

PCB engineering drawings mix sparse graphics, dense tables, and text whose meaning depends on page position. Localizing the regions and sending crops to specialized recognizers are determined as the methods for most parsers, so missed regions cannot be recovered downstream. We train a compact VLM to read the full page and get a sequence of region classes, normalized boxes, and text or HTML content. Bounding boxes are converted to coordinate tokens for supervision. Inference uses no detector or crop parser. The joint target is difficult to optimize because class and box tokens are sparse relative to the much longer content sequences. Our localization-first curriculum learns the class-box format before adding content targets with content-aware resampling. On the fixed validation split of the Engineering Drawing Dataset (ED dataset), Localization-First improves strict localization F1 by 0.0955 over joint training (paired image-bootstrap 95% interval: [0.0350, 0.1572]). G-Unified has the lowest NED, highest cell F1, and only nonzero exact-match score. It provides a detector-free baseline for full-page PCB drawing parsing.