# Symbol as Points: Panoptic Symbol Spotting via Point-based Representation

arXiv: https://arxiv.org/abs/2401.10556 · HTML: https://mlsanity.com/paper/2401.10556/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 28 of 28)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Mask3D: Mask Transformer for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2210.03105/) (2022)
- [DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation](https://mlsanity.com/paper/2007.11301/) (2020)
- [Im2Vec: Synthesizing Vector Graphics without Vector Supervision](https://mlsanity.com/paper/2102.02798/) (2021)
- [Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation](https://mlsanity.com/paper/2103.14962/) (2021)
- [Analyzing and Improving Representations with the Soft Nearest Neighbor Loss](https://mlsanity.com/paper/1902.01889/) (2019)
- [LiDAR-based Panoptic Segmentation via Dynamic Shifting Network](https://mlsanity.com/paper/2011.11964/) (2020)
- [Learning to predict crisp boundaries](https://mlsanity.com/paper/1807.10097/) (2018)
- [Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic Segmentation via Clustering Pseudo Heatmap](https://mlsanity.com/paper/2205.07002/) (2022)
- [ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution](https://mlsanity.com/paper/2303.00246/) (2023)
- [FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spotting](https://mlsanity.com/paper/2105.07147/) (2021)
- [Symbol Spotting on Digital Architectural Floor Plans Using a Deep Learning-based Framework](https://mlsanity.com/paper/2006.00684/) (2020)
- [Position-Guided Point Cloud Panoptic Segmentation Transformer](https://mlsanity.com/paper/2303.13509/) (2023)
- [GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawings](https://mlsanity.com/paper/2201.00625/) (2022)
- [Recognizing Vector Graphics without Rasterization](https://mlsanity.com/paper/2111.03281/) (2021)
- [RendNet: Unified 2D/3D Recognizer With Latent Space Rendering](https://mlsanity.com/paper/2206.10066/) (2022)

## Built on by (most cited first, 11 of 11)

- [SymPoint Revolutionized: Boosting Panoptic Symbol Spotting with Layer Feature Enhancement](https://mlsanity.com/paper/2407.01928/) (2024)
- [ArchCAD-400K: A Large-Scale CAD drawings Dataset and New Baseline for Panoptic Symbol Spotting](https://mlsanity.com/paper/2503.22346/) (2025)
- [CADSpotting: Robust Panoptic Symbol Spotting on Large-Scale CAD Drawings](https://mlsanity.com/paper/2412.07377/) (2024)
- [Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawings](https://mlsanity.com/paper/2505.23395/) (2025)
- [CAGE: Continuity-Aware edGE Network Unlocks Robust Floorplan Reconstruction](https://mlsanity.com/paper/2509.15459/) (2025)
- [Pixel-Wise Symbol Spotting via Progressive Points Location for Parsing CAD Images](https://mlsanity.com/paper/2404.10985/) (2024)
- [VectorGraphNET: Graph Attention Networks for Accurate Segmentation of Complex Technical Drawings](https://mlsanity.com/paper/2410.01336/) (2024)
- [Text-Enhanced Panoptic Symbol Spotting in CAD Drawings](https://mlsanity.com/paper/2510.11091/) (2025)
- [Text-Aided Multi-Modal Panoptic Symbol Spotting for CAD Floor Plan Drawings](https://mlsanity.com/paper/2607.12678/) (2026)
- [PolarSym: Polar Geometry-aware Attention for CAD Floorplan Parsing](https://mlsanity.com/paper/2608.11793/) (2026)
- [Training-Free Agentic Computer Vision for Structural Component Detection in 2D Structural Framing Plans](https://mlsanity.com/paper/2608.17237/) (2026)

## Abstract (arXiv)

This work studies the problem of panoptic symbol spotting, which is to spot and parse both countable object instances (windows, doors, tables, etc.) and uncountable stuff (wall, railing, etc.) from computer-aided design (CAD) drawings. Existing methods typically involve either rasterizing the vector graphics into images and using image-based methods for symbol spotting, or directly building graphs and using graph neural networks for symbol recognition. In this paper, we take a different approach, which treats graphic primitives as a set of 2D points that are locally connected and use point cloud segmentation methods to tackle it. Specifically, we utilize a point transformer to extract the primitive features and append a mask2former-like spotting head to predict the final output. To better use the local connection information of primitives and enhance their discriminability, we further propose the attention with connection module (ACM) and contrastive connection learning scheme (CCL). Finally, we propose a KNN interpolation mechanism for the mask attention module of the spotting head to better handle primitive mask downsampling, which is primitive-level in contrast to pixel-level for the image. Our approach, named SymPoint, is simple yet effective, outperforming recent state-of-the-art method GAT-CADNet by an absolute increase of 9.6% PQ and 10.4% RQ on the FloorPlanCAD dataset. The source code and models will be available at https://github.com/nicehuster/SymPoint.