mlsanityData to 7 Oct 2026

TART: Token-based Architecture Transformer for Neural Network Performance Prediction

arXiv 2501.02007 ↗First posted 2 Jan 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In the realm of neural architecture design, achieving high performance is largely reliant on the manual expertise of researchers. Despite the emergence of Neural Architecture Search (NAS) as a promising technique for automating this process, current NAS methods still require human input to expand the search space and cannot generate new architectures. This paper explores the potential of Transformers in comprehending neural architectures and their performance, with the objective of establishing the foundation for utilizing Transformers to generate novel networks. We propose the Token-based Architecture Transformer (TART), which predicts neural network performance without the need to train candidate networks. TART attains state-of-the-art performance on the DeepNets-1M dataset for performance prediction tasks without edge information, indicating the potential of Transformers to aid in discovering novel and high-performing neural architectures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 2 Jan 2025.

Builds on

21 of 29 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  5. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  6. 2018Invariant and Equivariant Graph Networkscited by 339
  7. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  8. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  9. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  10. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  11. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  12. 2018Graph HyperNetworks for Neural Architecture Searchcited by 131
9 more
  1. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  2. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  3. 2021Generative Adversarial Transformerscited by 120
  4. 2019Neural Predictor for Neural Architecture Searchcited by 116
  5. 2019AutoGAN: Neural Architecture Search for Generative Adversarial Networkscited by 94
  6. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  7. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  8. 2021Directed Acyclic Graph Neural Networkscited by 70
  9. 2021Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformerscited by 19

Built on by

Not cited yet.