mlsanityData to 7 Oct 2026

Understanding Survey Paper Taxonomy about Large Language Models via Graph Representation Learning

arXiv 2402.10409 ↗First posted 16 Feb 2024cs.CL cs.AI cs.IRcited by 5

Not on the map. Why?

Abstract (arXiv)

As new research on Large Language Models (LLMs) continues, it is difficult to keep up with new research and models. To help researchers synthesize the new research many have written survey papers, but even those have become numerous. In this paper, we develop a method to automatically assign survey papers to a taxonomy. We collect the metadata of 144 LLM survey papers and explore three paradigms to classify papers within the taxonomy. Our work indicates that leveraging graph structure information on co-category graphs can significantly outperform the language models in two paradigms; pre-trained language models' fine-tuning and zero-shot/few-shot classifications using LLMs. We find that our model surpasses an average human recognition level and that fine-tuning LLMs using weak labels generated by a smaller model, such as the GCN in this study, can be more effective than using ground-truth labels, revealing the potential of weak-to-strong generalization in the taxonomy classification task.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 22 of its 35 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

22 of 35 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  8. 2017Graph Attention Networkscited by 8,854
  9. 2017Inductive Representation Learning on Large Graphscited by 6,284
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  11. 2018How Powerful are Graph Neural Networks?cited by 4,355
  12. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
10 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  5. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  6. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  7. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  8. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  9. 2018Graph Convolutional Networks for Text Classificationcited by 345
  10. 2022Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  2. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  3. 2025Stop DDoS Attacking the Research Community with AI-Generated Survey Paperscited by 6
  4. 2024Enhanced Credit Score Prediction Using Ensemble Deep Learning Modelcited by 2
  5. 2026SurveyReview: A Reviewer-Aligned Benchmark for Survey Evaluatorscited by 0