mlsanityData to 14 Oct 2026

Automatic Graph Topology-Aware Transformer

arXiv 2405.19779 ↗First posted 30 May 2024cs.NE cs.GR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Existing efforts are dedicated to designing many topologies and graph-aware strategies for the graph Transformer, which greatly improve the model's representation capabilities. However, manually determining the suitable Transformer architecture for a specific graph dataset or task requires extensive expert knowledge and laborious trials. This paper proposes an evolutionary graph Transformer architecture search framework (EGTAS) to automate the construction of strong graph Transformers. We build a comprehensive graph Transformer search space with the micro-level and macro-level designs. EGTAS evolves graph Transformer topologies at the macro level and graph-aware strategies at the micro level. Furthermore, a surrogate model based on generic architectural coding is proposed to directly predict the performance of graph Transformers, substantially reducing the evaluation cost of evolutionary search. We demonstrate the efficacy of EGTAS across a range of graph-level and node-level tasks, encompassing both small-scale and large-scale graph datasets. Experimental results and ablation studies show that EGTAS can construct high-performance architectures that rival state-of-the-art manual and automated baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 56 of its 94 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

56 of 94 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  9. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  11. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
  12. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
38 more
  1. 2020Simple and Deep Graph Convolutional Networkscited by 860
  2. 2021How Attentive are Graph Attention Networks?cited by 835
  3. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  4. 2020Efficient Transformers: A Surveycited by 790
  5. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 787
  6. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  7. 2020Benchmarking Graph Neural Networkscited by 680
  8. 2020A Generalization of Transformer Networks to Graphscited by 661
  9. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  10. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  11. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  12. 2017Residual Gated Graph ConvNetscited by 346
  13. 2021Mesh Graphormercited by 265
  14. 2017A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applicationscited by 260
  15. 2021Weisfeiler and Lehman Go Cellular: CW Networkscited by 218
  16. 2022Representing Long-Range Context for Graph Neural Networks with Global Attentioncited by 195
  17. 2022Structure-Aware Transformer for Graph Representation Learningcited by 194
  18. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 177
  19. 2022Graph Neural Networks for Graphs with Heterophily: A Surveycited by 175
  20. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  21. 2021From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awarenesscited by 155
  22. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  23. 2020Directional Graph Networkscited by 133
  24. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 114
  25. 2022Transformer for Graphs: An Overview from Architecture Perspectivecited by 107
  26. 2023Attending to Graph Transformerscited by 83
  27. 2019Auto-GNN: Neural Architecture Search of Graph Neural Networkscited by 74
  28. 2021Gophormer: Ego-Graph Transformer for Node Classificationcited by 63
  29. 2023Video Event Restoration Based on Keyframes for Video Anomaly Detectioncited by 54
  30. 2021Search to aggregate neighborhood for graph neural networkcited by 48
  31. 2021Walking Out of the Weisfeiler Leman Hierarchy: Graph Learning Beyond Message Passingcited by 47
  32. 2021Automated Machine Learning on Graphs: A Surveycited by 39
  33. 2021Pooling Architecture Search for Graph Classificationcited by 36
  34. 2020Simplifying Architecture Search for Graph Neural Networkcited by 28
  35. 2022NAS-Bench-Graph: Benchmarking Graph Neural Architecture Searchcited by 25
  36. 2023Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Modelcited by 25
  37. 2021Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspectivecited by 19
  38. 2020Graph Neural Network Architecture Search for Molecular Property Predictioncited by 17

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Graph Transformers: A Surveycited by 32
  2. 2024Knowledge-aware Evolutionary Graph Neural Architecture Searchcited by 3
  3. 2025SA-GAT-SR: Self-Adaptable Graph Attention Networks with Symbolic Regression for high-fidelity material property predictioncited by 0
  4. 2025DARTS-GT: Differentiable Architecture Search for Graph Transformers with Quantifiable Instance-Specific Interpretability Analysiscited by 0