mlsanityData to 14 Oct 2026

Scale-Free Graph-Language Models

arXiv 2502.15189 ↗First posted 21 Feb 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Graph-language models (GLMs) have demonstrated great potential in graph-based semi-supervised learning. A typical GLM consists of two key stages: graph generation and text embedding, which are usually implemented by inferring a latent graph and finetuning a language model (LM), respectively. However, the former often relies on artificial assumptions about the underlying edge distribution, while the latter requires extensive data annotations. To tackle these challenges, this paper introduces a novel GLM that integrates graph generation and text embedding within a unified framework. Specifically, for graph generation, we leverage an inherent characteristic of real edge distribution--the scale-free property--as a structural prior. We unexpectedly find that this natural property can be effectively approximated by a simple k-nearest neighbor (KNN) graph. For text embedding, we develop a graph-based pseudo-labeler that utilizes scale-free graphs to provide complementary supervision for improved LM finetuning. Extensive experiments on representative datasets validate our findings on the scale-free structural approximation of KNN graphs and demonstrate the effectiveness of integrating graph generation and text embedding with a real structural prior. Our code is available at https://github.com/Jianglin954/SFGL.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

19 of 33 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Graph Attention Networkscited by 8,923
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  9. 2023Improving Text Embeddings with Large Language Modelscited by 303
  10. 2019Learning Discrete Structures for Graph Neural Networkscited by 231
  11. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  12. 2021SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networkscited by 110
7 more
  1. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 85
  2. 2024Making Text Embedders Few-Shot Learnerscited by 79
  3. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
  4. 2018Subspace Clustering by Block Diagonal Representationcited by 38
  5. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  6. 2023Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphscited by 23
  7. 2023Latent Graph Inference with Limited Supervisioncited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Indra Representation Hypothesis for Multimodal Alignmentcited by 4
  2. 2026A Unified Perspective for Learning Graph Representations Across Multi-Level Abstractionscited by 0