mlsanityData to 14 Oct 2026

Learning to Ground Before Reading: Unified PCB Engineering Drawing Parsing with Compact Vision-Language Models

arXiv 2608.29268 ↗First posted 29 Aug 2026cs.CV cs.MMcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

PCB engineering drawings mix sparse graphics, dense tables, and text whose meaning depends on page position. Localizing the regions and sending crops to specialized recognizers are determined as the methods for most parsers, so missed regions cannot be recovered downstream. We train a compact VLM to read the full page and get a sequence of region classes, normalized boxes, and text or HTML content. Bounding boxes are converted to coordinate tokens for supervision. Inference uses no detector or crop parser. The joint target is difficult to optimize because class and box tokens are sparse relative to the much longer content sequences. Our localization-first curriculum learns the class-box format before adding content targets with content-aware resampling. On the fixed validation split of the Engineering Drawing Dataset (ED dataset), Localization-First improves strict localization F1 by 0.0955 over joint training (paired image-bootstrap 95% interval: [0.0350, 0.1572]). G-Unified has the lowest NED, highest cell F1, and only nonzero exact-match score. It provides a detector-free baseline for full-page PCB drawing parsing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 26 references that are in the map. First posted on arXiv 29 Aug 2026.

Builds on

20 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction6 Nov
  2. 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design21 Nov

Other references · 18

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  6. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  7. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  8. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  9. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  10. 2021OCR-free Document Understanding Transformercited by 407
  11. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  12. 2019Image-based table recognition: data, model, and evaluationcited by 237
6 more
  1. 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 148
  2. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 126
  3. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 116
  4. 2021LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysiscited by 58
  5. 2024UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognitioncited by 11
  6. 2025From Drawings to Decisions: A Hybrid Vision-Language Framework for Parsing 2D Engineering Drawings into Structured Manufacturing Knowledgecited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Model
  2. 2026UniPCB: A Unified Vision-Language Benchmark for Open-Ended PCB Quality Inspection
  3. 2026Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding
  4. 2025Ground-V: Teaching VLMs to Ground Complex Instructions in Pixels
  5. 2026Focus, Don't Prune: Identifying Instruction-Relevant Regions for Information-Rich Image Understanding
  6. 2026Benchmarking Deep Learning Approaches for AEC Engineering Drawing Layout Detection and Information Extraction
  7. 2026AECV-Bench: Benchmarking Multimodal Models on Architectural and Engineering Drawings Understanding
  8. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding