mlsanityData to 14 Oct 2026

Proficient Graph Neural Network Design by Accumulating Knowledge on Large Language Models

arXiv 2408.06717 ↗First posted 13 Aug 2024cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

High-level automation is increasingly critical in AI, driven by rapid advances in large language models (LLMs) and AI agents. However, LLMs, despite their general reasoning power, struggle significantly in specialized, data-sensitive tasks such as designing Graph Neural Networks (GNNs). This difficulty arises from (1) the inherent knowledge gaps in modeling the intricate, varying relationships between graph properties and suitable architectures and (2) the external noise from misleading descriptive inputs, often resulting in generic or even misleading model suggestions. Achieving proficiency in designing data-aware models -- defined as the meta-level capability to systematically accumulate, interpret, and apply data-specific design knowledge -- remains challenging for existing automated approaches, due to their inefficient construction and application of meta-knowledge. To achieve meta-level proficiency, we propose DesiGNN, a knowledge-centered framework that systematically converts past model design experience into structured, fine-grained knowledge priors well-suited for meta-learning with LLMs. To account for the inherent variability and external noise, DesiGNN aligns empirical property filtering from extensive benchmarks with adaptive elicitation of literature insights via LLMs. By constructing a solid meta-knowledge between unseen graph understanding and known effective architecture patterns, DesiGNN can deliver top-5.77% initial model proposals for unseen datasets within seconds and achieve consistently superior performance with minimal search cost compared to baselines.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 51 of its 77 references that are in the map. First posted on arXiv 13 Aug 2024.

Builds on

51 of 77 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2017Graph Attention Networkscited by 8,923
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2018How Powerful are Graph Neural Networks?cited by 4,393
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  10. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
38 more
  1. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  2. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  3. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  4. 2023Large Language Models as Optimizerscited by 864
  5. 2018Link Prediction Based on Graph Neural Networkscited by 832
  6. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  7. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  8. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 335
  9. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  10. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  11. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  12. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  13. 2019Graph Neural Networks with convolutional ARMA filterscited by 185
  14. 2020Design Space for Graph Neural Networkscited by 182
  15. 2021Meta-learning via Language Model In-context Tuningcited by 164
  16. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 125
  17. 2019Neural Predictor for Neural Architecture Searchcited by 118
  18. 2023Mindstorms in Natural Language-Based Societies of Mindcited by 107
  19. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  20. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  21. 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 81
  22. 2019Auto-GNN: Neural Architecture Search of Graph Neural Networkscited by 74
  23. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  24. 2023Is GPT-4 a Good Data Analyst?cited by 70
  25. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  26. 2021Search to aggregate neighborhood for graph neural networkcited by 48
  27. 2019GraphNAS: Graph Neural Architecture Search with Reinforcement Learningcited by 41
  28. 2021Pooling Architecture Search for Graph Classificationcited by 36
  29. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  30. 2023LLM Performance Predictors are good initializers for Architecture Searchcited by 29
  31. 2022NAS-Bench-Graph: Benchmarking Graph Neural Architecture Searchcited by 25
  32. 2021AutoGEL: An Automated Graph Neural Network with Explicit Link Informationcited by 17
  33. 2021Searching to Sparsify Tensor Decomposition for N-ary Relational Datacited by 13
  34. 2023Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Taskscited by 13
  35. 2022Dual-Geometric Space Embedding Model for Two-View Knowledge Graphscited by 12
  36. 2022Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directionscited by 9
  37. 2023AutoTransfer: AutoML with Knowledge Transfer -- An Application to Graph Neural Networkscited by 8
  38. 2021Fisher Task Distance and Its Application in Neural Architecture Searchcited by 7

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Automated Decision-Making on Networks with LLMs through Knowledge-Guided Evolution
  2. 2026LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems
  3. 2023Graph Neural Prompting with Large Language Models
  4. 2024Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining
  5. 2025Are LLMs Better GNN Helpers? Rethinking Robust Graph Learning under Deficiencies with Iterative Refinement
  6. 2026Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning
  7. 2025Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models
  8. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation