# DeepGate3: Towards Scalable Circuit Representation Learning

arXiv: https://arxiv.org/abs/2407.11095 · HTML: https://mlsanity.com/paper/2407.11095/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022)
- [Understanding over-squashing and bottlenecks on graphs via curvature](https://mlsanity.com/paper/2111.14522/) (2021)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Blockwise Self-Attention for Long Document Understanding](https://mlsanity.com/paper/1911.02972/) (2019)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [Benchmarking Large Language Models for Automated Verilog RTL Code Generation](https://mlsanity.com/paper/2212.11140/) (2022)
- [OpenGraph: Towards Open Graph Foundation Models](https://mlsanity.com/paper/2403.01121/) (2024)
- [Data Scaling Laws in NMT: The Effect of Noise and Architecture](https://mlsanity.com/paper/2202.01994/) (2022)
- [DeepGate: Learning Neural Representations of Logic Gates](https://mlsanity.com/paper/2111.14616/) (2021)
- [DeepGate2: Functionality-Aware Circuit Representation Learning](https://mlsanity.com/paper/2305.16373/) (2023)
- [Gamora: Graph Learning based Symbolic Reasoning for Large-Scale Boolean Networks](https://mlsanity.com/paper/2303.08256/) (2023)
- [PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs](https://mlsanity.com/paper/2203.10304/) (2022)
- [DeepSAT: An EDA-Driven Learning Framework for SAT](https://mlsanity.com/paper/2205.13745/) (2022)
- [DeepTPI: Test Point Insertion with Deep Reinforcement Learning](https://mlsanity.com/paper/2206.06975/) (2022)
- [DeepSeq: Deep Sequential Circuit Learning](https://mlsanity.com/paper/2302.13608/) (2023)

## Built on by (most cited first, 26 of 26)

- [DeepCircuitX: A Comprehensive Repository-Level Dataset for RTL Code Understanding, Generation, and PPA Analysis](https://mlsanity.com/paper/2502.18297/) (2025)
- [DeepGate4: Efficient and Effective Representation Learning for Circuit Design at Scale](https://mlsanity.com/paper/2502.01681/) (2025)
- [NetTAG: A Multimodal RTL-and-Layout-Aligned Netlist Foundation Model via Text-Attributed Graph](https://mlsanity.com/paper/2504.09260/) (2025)
- [Revolution or Hype? Seeking the Limits of Large Models in Hardware Design](https://mlsanity.com/paper/2509.04905/) (2025)
- [Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignment](https://mlsanity.com/paper/2502.12732/) (2025)
- [MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs](https://mlsanity.com/paper/2507.19525/) (2025)
- [DeepCell: Self-Supervised Multiview Fusion for Circuit Representation Learning](https://mlsanity.com/paper/2502.06816/) (2025)
- [DICE: Device-level Integrated Circuits Encoder with Graph Contrastive Pretraining](https://mlsanity.com/paper/2502.08949/) (2025)
- [Circuit-Aware SAT Solving: Guiding CDCL via Conditional Probabilities](https://mlsanity.com/paper/2508.04235/) (2025)
- [GenEDA: Towards Generative Netlist Functional Reasoning via Cross-Modal Circuit Encoder-Decoder Alignment](https://mlsanity.com/paper/2504.09485/) (2025)
- [Functional Matching of Logic Subgraphs: Beyond Structural Isomorphism](https://mlsanity.com/paper/2505.21988/) (2025)
- [FuncGNN: Learning Functional Semantics of Logic Circuits with Graph Neural Networks](https://mlsanity.com/paper/2506.06787/) (2025)
- [Alignment Unlocks Complementarity: A Framework for Multiview Circuit Representation Learning](https://mlsanity.com/paper/2509.20968/) (2025)
- [R2G: A Multi-View Circuit Graph Benchmark Suite from RTL to GDSII](https://mlsanity.com/paper/2604.08810/) (2026)
- [Modeling Relational Logic Circuits for And-Inverter Graph Convolutional Network](https://mlsanity.com/paper/2508.11991/) (2025)
- [TRACE: Learning to Compute on Circuit Graphs](https://mlsanity.com/paper/2509.21886/) (2025)
- [Learning Representations Through Contrastive Neural Model Checking](https://mlsanity.com/paper/2510.01853/) (2025)
- [BoolSkeleton: Boolean Network Skeletonization via Homogeneous Pattern Reduction](https://mlsanity.com/paper/2511.02196/) (2025)
- [InF-ATPG: Intelligent FFR-Driven ATPG with Advanced Circuit Representation Guided Reinforcement Learning](https://mlsanity.com/paper/2512.00079/) (2025)
- [ReVEAL: GNN-Guided Reverse Engineering for Formal Verification of Optimized Multipliers](https://mlsanity.com/paper/2512.22260/) (2025)
- [GNN-based Path-aware multi-view Circuit Learning for Technology Mapping](https://mlsanity.com/paper/2601.14286/) (2026)
- [Wrong Code, Right Structure: Learning Netlist Representations from Imperfect LLM-Generated RTL](https://mlsanity.com/paper/2603.09161/) (2026)
- [KCLNet: Electrically Equivalence-Oriented Graph Representation Learning for Analog Circuits](https://mlsanity.com/paper/2603.24101/) (2026)
- [Graph Computation Meets Circuit Algebra: A Task-Aligned Analysis of Graph Neural Networks for Electronic Design Automation](https://mlsanity.com/paper/2605.08291/) (2026)
- [Beyond Flat Netlist: Hierarchical Graph Representation Learning for Scalable Analysis of Sequential Circuits](https://mlsanity.com/paper/2608.28188/) (2026)
- [ReDIL-GNN: Resynthesis Domain Incremental Learning for Circuit Graph Neural Networks](https://mlsanity.com/paper/2609.18595/) (2026)

## Abstract (arXiv)

Circuit representation learning has shown promising results in advancing the field of Electronic Design Automation (EDA). Existing models, such as DeepGate Family, primarily utilize Graph Neural Networks (GNNs) to encode circuit netlists into gate-level embeddings. However, the scalability of GNN-based models is fundamentally constrained by architectural limitations, impacting their ability to generalize across diverse and complex circuit designs. To address these challenges, we introduce DeepGate3, an enhanced architecture that integrates Transformer modules following the initial GNN processing. This novel architecture not only retains the robust gate-level representation capabilities of its predecessor, DeepGate2, but also enhances them with the ability to model subcircuits through a novel pooling transformer mechanism. DeepGate3 is further refined with multiple innovative supervision tasks, significantly enhancing its learning process and enabling superior representation of both gate-level and subcircuit structures. Our experiments demonstrate marked improvements in scalability and generalizability over traditional GNN-based approaches, establishing a significant step forward in circuit representation learning technology.