mlsanityData to 14 Oct 2026

FoGE: Fock Space inspired encoding for graph prompting

arXiv 2507.02937 ↗First posted 26 Jun 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent results show that modern Large Language Models (LLM) are indeed capable of understanding and answering questions about structured data such as graphs. This new paradigm can lead to solutions that require less supervision while, at the same time, providing a model that can generalize and answer questions beyond the training labels. Existing proposals often use some description of the graph to create an ``augmented'' prompt fed to the LLM. For a chosen class of graphs, if a well-tailored graph encoder is deployed to play together with a pre-trained LLM, the model can answer graph-related questions well. Existing solutions to graph-based prompts range from graph serialization to graph transformers. In this work, we show that the use of a parameter-free graph encoder based on Fock space representations, a concept borrowed from mathematical physics, is remarkably versatile in this problem setting. The simple construction, inherited directly from the theory with a few small adjustments, can provide rich and informative graph encodings, for a wide range of different graphs. We investigate the use of this idea for prefix-tuned prompts leveraging the capabilities of a pre-trained, frozen LLM. The modifications lead to a model that can answer graph-related questions -- from simple graphs to proteins to hypergraphs -- effectively and with minimal, if any, adjustments to the architecture. Our work significantly simplifies existing solutions and generalizes well to multiple different graph-based structures effortlessly.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 50 of its 87 references that are in the map. First posted on arXiv 26 Jun 2025.

Builds on

50 of 87 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2017Graph Attention Networkscited by 8,923
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  10. 2022Emergent Abilities of Large Language Modelscited by 2,512
  11. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
38 more
  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  3. 2018Deep Graph Infomaxcited by 1,106
  4. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  5. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  6. 2020Simple and Deep Graph Convolutional Networkscited by 860
  7. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  8. 2020A Generalization of Transformer Networks to Graphscited by 661
  9. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  10. 2020Deep Graph Contrastive Representation Learningcited by 539
  11. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  12. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  13. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 267
  14. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  15. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  16. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  17. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  18. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 176
  19. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 167
  20. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 118
  21. 2023Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Studycited by 116
  22. 2018Large-Scale Learnable Graph Convolutional Networkscited by 113
  23. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Modelcited by 107
  24. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 105
  25. 2020A comparison of Vector Symbolic Architecturescited by 64
  26. 2023Text-To-Concept (and Back) via Cross-Model Alignmentcited by 64
  27. 2023Graph Neural Prompting with Large Language Modelscited by 64
  28. 2023Clifford Group Equivariant Neural Networkscited by 55
  29. 2024Large Language Model for Table Processing: A Surveycited by 52
  30. 2019Quaternion Convolutional Neural Networkscited by 48
  31. 2023Geometric Algebra Transformercited by 48
  32. 2023Geometric Clifford Algebra Networkscited by 38
  33. 2021Learning with Holographic Reduced Representationscited by 25
  34. 2024Clifford-Steerable Convolutional Neural Networkscited by 25
  35. 2021Graph Transformer Networks: Learning Meta-path Graphs to Improve GNNscited by 22
  36. 2023Recasting Self-Attention with Holographic Reduced Representationscited by 14
  37. 2021VolterraNet: A higher order convolutional network with group equivariance for homogeneous manifoldscited by 11
  38. 2023Machine Learning Clifford invariants of ADE Coxeter elementscited by 6

Built on by

Not cited yet.