mlsanityData to 14 Oct 2026

Probing Graph Representations

arXiv 2303.03951 ↗First posted 7 Mar 2023cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Today we have a good theoretical understanding of the representational power of Graph Neural Networks (GNNs). For example, their limitations have been characterized in relation to a hierarchy of Weisfeiler-Lehman (WL) isomorphism tests. However, we do not know what is encoded in the learned representations. This is our main question. We answer it using a probing framework to quantify the amount of meaningful information captured in graph representations. Our findings on molecular datasets show the potential of probing for understanding the inductive biases of graph-based models. We compare different families of models and show that transformer-based models capture more chemically relevant information compared to models based on message passing. We also study the effect of different design choices such as skip connections and virtual nodes. We advocate for probing as a useful diagnostic tool for evaluating graph-based models.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

39 of 50 references.

References · 39

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2018How Powerful are Graph Neural Networks?cited by 4,393
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  9. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  10. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  11. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  12. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 631
27 more
  1. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  2. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 384
  3. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  4. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  5. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  6. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  7. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  8. 2020XGNN: Towards Model-Level Explanations of Graph Neural Networkscited by 211
  9. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
  10. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  11. 2020A Theory of Usable Information Under Computational Constraintscited by 162
  12. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  13. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  14. 2021CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networkscited by 119
  15. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  16. 2021ProtGNN: Towards Self-Explaining Graph Neural Networkscited by 96
  17. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 70
  18. 2021Conditional probing: measuring usable information beyond a baselinecited by 62
  19. 2020Pareto Probing: Trading Off Accuracy for Complexitycited by 55
  20. 2020Asking without Telling: Exploring Latent Ontologies in Contextual Representationscited by 34
  21. 2021MEG: Generating Molecular Counterfactual Explanations for Deep Graph Networkscited by 32
  22. 2021DirectProbe: Studying Representations without Classifierscited by 25
  23. 2021A Bayesian Framework for Information-Theoretic Probingcited by 22
  24. 2022Evaluating Self-Supervised Learning for Molecular Graph Embeddingscited by 22
  25. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
  26. 2021Probing artificial neural networks: insights from neurosciencecited by 17
  27. 2019Probing the State of the Art: A Critical Look at Visual Representation Evaluationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 1; 2026: 1.

10 papers.

  1. 2024Node Identifiers: Compact, Discrete Representations for Efficient Graph Learningcited by 10
  2. 2023Improving Self-supervised Molecular Representation Learning using Persistent Homologycited by 6
  3. 2023Substance or Style: What Does Your Image Embedding Know?cited by 5
  4. 2024Descriptive Kernel Convolution Network with Improved Random Walk Kernelcited by 5
  5. 2023Transformers are efficient hierarchical chemical graph learnerscited by 4
  6. 2024NetInfoF Framework: Measuring and Exploiting Network Usable Informationcited by 4
  7. 2025Superposition in Graph Neural Networkscited by 2
  8. 2024MoleculeCLA: Rethinking Molecular Benchmark via Computational Ligand-Target Binding Analysiscited by 0
  9. 2024Do graph neural network states contain graph properties?cited by 0
  10. 2026Disentangling Homophily and Rarity: Explaining Failure in Graph Neural Networkscited by 0