# Text-Aided Multi-Modal Panoptic Symbol Spotting for CAD Floor Plan Drawings

arXiv: https://arxiv.org/abs/2607.12678 · HTML: https://mlsanity.com/paper/2607.12678/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 46 references that are in the map.
- First posted on arXiv 14 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [A Survey on Model Compression for Large Language Models](https://mlsanity.com/paper/2308.07633/) (2023)
- [CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis](https://mlsanity.com/paper/1904.01920/) (2019)
- [FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spotting](https://mlsanity.com/paper/2105.07147/) (2021)
- [Symbol Spotting on Digital Architectural Floor Plans Using a Deep Learning-based Framework](https://mlsanity.com/paper/2006.00684/) (2020)
- [GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawings](https://mlsanity.com/paper/2201.00625/) (2022)
- [Recognizing Vector Graphics without Rasterization](https://mlsanity.com/paper/2111.03281/) (2021)
- [Symbol as Points: Panoptic Symbol Spotting via Point-based Representation](https://mlsanity.com/paper/2401.10556/) (2024)
- [SymPoint Revolutionized: Boosting Panoptic Symbol Spotting with Layer Feature Enhancement](https://mlsanity.com/paper/2407.01928/) (2024)
- [ArchCAD-400K: A Large-Scale CAD drawings Dataset and New Baseline for Panoptic Symbol Spotting](https://mlsanity.com/paper/2503.22346/) (2025)
- [CADSpotting: Robust Panoptic Symbol Spotting on Large-Scale CAD Drawings](https://mlsanity.com/paper/2412.07377/) (2024)
- [Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawings](https://mlsanity.com/paper/2505.23395/) (2025)
- [Pixel-Wise Symbol Spotting via Progressive Points Location for Parsing CAD Images](https://mlsanity.com/paper/2404.10985/) (2024)
- [VectorGraphNET: Graph Attention Networks for Accurate Segmentation of Complex Technical Drawings](https://mlsanity.com/paper/2410.01336/) (2024)

## Abstract (arXiv)

Computer-Aided Design (CAD) floor plan drawings contain both graphical primitives and textual annotations, which provide complementary geometric and semantic cues for intelligent design understanding. Among CAD analysis tasks, panoptic symbol spotting has become increasingly important with the growing demand for industrial digitalization and deep learning-based automation. However, most existing methods remain primarily primitive-centric and underexploit textual annotations, despite their critical semantic value. Even the few text-aware approaches often treat annotations only superficially, without properly modeling complex syntax and hierarchical semantics of CAD annotations, which leads to semantic loss and suboptimal spotting performance. To address these limitations, we propose TextCAD, a multimodal framework that jointly models graphical primitives and textual annotations for panoptic symbol spotting. Specifically, we design a Type-Attribute Correlation Encoder (TACE) to explicitly encode the compositional semantics within annotations by jointly modeling their types and attributes. We further introduce a Semantic Hierarchy Alignment framework with Multi-level Semantic Filtering (MSF) and primitive downsampling, which adaptively aligns annotation semantics with graphical primitives at different semantic levels and enables accurate cross-modal semantic injection and fusion. Experiments on real-world building-design datasets show that TextCAD effectively improves symbol spotting performance and achieves state-of-the-art results.