mlsanityData to 7 Oct 2026

FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer

arXiv 2403.12821 ↗First posted 19 Mar 2024cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

The success of a specific neural network architecture is closely tied to the dataset and task it tackles; there is no one-size-fits-all solution. Thus, considerable efforts have been made to quickly and accurately estimate the performances of neural architectures, without full training or evaluation, for given tasks and datasets. Neural architecture encoding has played a crucial role in the estimation, and graphbased methods, which treat an architecture as a graph, have shown prominent performance. For enhanced representation learning of neural architectures, we introduce FlowerFormer, a powerful graph transformer that incorporates the information flows within a neural architecture. FlowerFormer consists of two key components: (a) bidirectional asynchronous message passing, inspired by the flows; (b) global attention built on flow-based masking. Our extensive experiments demonstrate the superiority of FlowerFormer over existing neural encoding methods, and its effectiveness extends beyond computer vision models to include graph neural networks and auto speech recognition models. Our code is available at http://github.com/y0ngjaenius/CVPR2024_FLOWERFormer.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 35 of its 55 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

35 of 55 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  6. 2020A Generalization of Transformer Networks to Graphscited by 653
  7. 2017Progressive Neural Architecture Searchcited by 483
  8. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  9. 2017Residual Gated Graph ConvNetscited by 341
  10. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  11. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  12. 2018Neural Architecture Optimizationcited by 164
23 more
  1. 2023Graph Inductive Biases in Transformers without Message Passingcited by 150
  2. 2021GraphiT: Encoding Graph Structure in Transformerscited by 146
  3. 2019Neural Predictor for Neural Architecture Searchcited by 116
  4. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  5. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  6. 2022A graph-transformer for whole slide image classificationcited by 84
  7. 2021Directed Acyclic Graph Neural Networkscited by 70
  8. 2020Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarkscited by 67
  9. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  10. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 63
  11. 2020A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAScited by 53
  12. 2020Semi-Supervised Neural Architecture Searchcited by 45
  13. 2019AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Searchcited by 33
  14. 2021Contrastive Neural Architecture Search with Neural Architecture Comparatorscited by 33
  15. 2022IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognitioncited by 31
  16. 2022NAS-Bench-Graph: Benchmarking Graph Neural Architecture Searchcited by 25
  17. 2022Transformers over Directed Acyclic Graphscited by 23
  18. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
  19. 2019Improving Graph Neural Network Representations of Logical Formulae with Subgraph Poolingcited by 20
  20. 2022NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Predictioncited by 12
  21. 2020NASGEM: Neural Architecture Search via Graph Embedding Methodcited by 9
  22. 2023Graph Propagation Transformer for Graph Representation Learningcited by 9
  23. 2021Graph-based Neural Architecture Search with Operation Embeddingscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025Multi-View Encoders for Performance Prediction in LLM-Based Agentic Workflowscited by 3
  2. 2025CARL: Causality-guided Architecture Representation Learning for an Interpretable Performance Predictorcited by 2
  3. 2025NN-Former: Rethinking Graph Structure in Neural Architecture Representationcited by 2
  4. 2025Simple Path Structural Encoding for Graph Transformerscited by 1
  5. 2025Loss Functions for Predictor-based Neural Architecture Searchcited by 1
  6. 2025ONNX-Net: Towards Universal Representations and Instant Performance Prediction for Neural Architecturescited by 1
  7. 2025Learning to Flow from Generative Pretext Tasks for Neural Architecture Encodingcited by 1
  8. 2026How Much Does Message Passing Matter? A Drop-In Study of GNN Layers for Neural Network Graph Regressioncited by 0
  9. 2026RiPPLE: Cross-Space Performance Prediction from Early Training for Neural Architecture Searchcited by 0