# Bringing Attention to CAD: Boundary Representation Learning via Transformer

arXiv: https://arxiv.org/abs/2504.07134 · HTML: https://mlsanity.com/paper/2504.07134/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [MeshNet: Mesh Neural Network for 3D Shape Representation](https://mlsanity.com/paper/1811.11424/) (2018)
- [BRepNet: A topological message passing system for solid models](https://mlsanity.com/paper/2104.00706/) (2021)
- [MeshCNN: A Network with an Edge](https://mlsanity.com/paper/1809.05910/) (2018)
- [Subdivision-Based Mesh Convolution Networks](https://mlsanity.com/paper/2106.02285/) (2021)
- [MeshWalker: Deep Mesh Understanding by Random Walks](https://mlsanity.com/paper/2006.05353/) (2020)
- ["FabSearch" : A 3D CAD Model Based Search Engine for Sourcing Manufacturing Services](https://mlsanity.com/paper/1809.06329/) (2018)
- [Language-based Photo Color Adjustment for Graphic Designs](https://mlsanity.com/paper/2308.03059/) (2023)
- [Intelligent CAD 2.0](https://mlsanity.com/paper/2410.03759/) (2024)
- [Push-pull direct modeling of solid CAD models](https://mlsanity.com/paper/1811.04036/) (2018)
- [Parametric/direct CAD integration](https://mlsanity.com/paper/2203.02252/) (2022)

## Built on by (most cited first, 9 of 9)

- [FoV-Net: Rotation-Invariant CAD B-rep Learning via Field-of-View Ray Casting](https://mlsanity.com/paper/2602.24084/) (2026)
- [Masked BRep Autoencoder via Hierarchical Graph Transformer](https://mlsanity.com/paper/2603.14927/) (2026)
- [BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding](https://mlsanity.com/paper/2606.05515/) (2026)
- [CADENA: Stepwise CAD Reverse Engineering](https://mlsanity.com/paper/2608.00799/) (2026)
- [Brep2Shape: Boundary and Shape Representation Alignment via Self-Supervised Transformers](https://mlsanity.com/paper/2602.07429/) (2026)
- [AirfoilGen: A valid-by-construction and performance-aware latent diffusion model for airfoil generation](https://mlsanity.com/paper/2605.20303/) (2026)
- [KDH-CAD: Knowledge-data hybrid CAD learning under data scarcity](https://mlsanity.com/paper/2606.01702/) (2026)
- [MidSurfNet: Learning Face Pairing for Mid-surface Abstraction of Thin-walled CAD Models](https://mlsanity.com/paper/2606.01891/) (2026)
- [Automatic assembly modeling through attention-based assembly feature mate prediction and assembly feature pattern recognition](https://mlsanity.com/paper/2610.14633/) (2026)

## Abstract (arXiv)

The recent rise of generative artificial intelligence (AI), powered by Transformer networks, has achieved remarkable success in natural language processing, computer vision, and graphics. However, the application of Transformers in computer-aided design (CAD), particularly for processing boundary representation (B-rep) models, remains largely unexplored. To bridge this gap, we propose a novel approach for adapting Transformers to B-rep learning, called the Boundary Representation Transformer (BRT). B-rep models pose unique challenges due to their irregular topology and continuous geometric definitions, which are fundamentally different from the structured and discrete data Transformers are designed for. To address this, BRT proposes a continuous geometric embedding method that encodes B-rep surfaces (trimmed and untrimmed) into Bezier triangles, preserving their shape and continuity without discretization. Additionally, BRT employs a topology-aware embedding method that organizes these geometric embeddings into a sequence of discrete tokens suitable for Transformers, capturing both geometric and topological characteristics within B-rep models. This enables the Transformer's attention mechanism to effectively learn shape patterns and contextual semantics of boundary elements in a B-rep model. Extensive experiments demonstrate that BRT achieves state-of-the-art performance in part classification and feature recognition tasks.