mlsanityData to 7 Oct 2026

Relational inductive biases, deep learning, and graph networks

arXiv 1806.01261 ↗First posted 4 Jun 2018cs.LG cs.AI stat.MLcited by 1,341

Not on the map. Why?

Cited by 36 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hallmark of human intelligence from infancy--remains a formidable challenge for modern AI. The following is part position paper, part review, and part unification. We argue that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective. Just as biology uses nature and nurture cooperatively, we reject the false choice between "hand-engineering" and "end-to-end" learning, and instead advocate for an approach which benefits from their complementary strengths. We explore how using relational inductive biases within deep learning architectures can facilitate learning about entities, relations, and rules for composing them. We present a new building block for the AI toolkit with a strong relational inductive bias--the graph network--which generalizes and extends various approaches for neural networks that operate on graphs, and provides a straightforward interface for manipulating structured knowledge and producing structured behaviors. We discuss how graph networks can support relational reasoning and combinatorial generalization, laying the foundation for more sophisticated, interpretable, and flexible patterns of reasoning. As a companion to this paper, we have released an open-source software library for building graph networks, with demonstrations of how to use them in practice.

Key facts. Cited by 1,341 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 1; 2021: 261; 2022: 305; 2023: 262; 2024: 175; 2025: 175; 2026: 160.

1,341 papers.

Sort:

By area:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  3. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  4. 2021Machine learning accelerated computational fluid dynamicscited by 461
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2021Neural Operator: Learning Maps Between Function Spacescited by 413
  7. 2021A Survey of Transformerscited by 399
  8. 2022Message Passing Neural PDE Solverscited by 368
  9. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 309
  10. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  11. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
  12. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
38 more
  1. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 225
  2. 2022Neural Networks and the Chomsky Hierarchycited by 214
  3. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
  4. 2021Towards Causal Representation Learningcited by 207
  5. 2021Identity-aware Graph Neural Networkscited by 197
  6. 2022Representing Long-Range Context for Graph Neural Networks with Global Attentioncited by 193
  7. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 188
  8. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  9. 2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
  10. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  11. 2022Revisiting Heterophily For Graph Neural Networkscited by 169
  12. 2021Spherical Message Passing for 3D Graph Networkscited by 166
  13. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 166
  14. 2022Forecasting Global Weather with Graph Neural Networkscited by 165
  15. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  16. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  17. 2024The Road Less Scheduledcited by 158
  18. 2023Towards Better Dynamic Graph Learning: New Architecture and Unified Librarycited by 154
  19. 2021Graph Condensation for Graph Neural Networkscited by 150
  20. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  21. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 145
  22. 2022ROLAND: Graph Learning Framework for Dynamic Graphscited by 131
  23. 2021ETA Prediction with Graph Neural Networks in Google Mapscited by 130
  24. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  25. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  26. 2021Vision Transformer for Small-Size Datasetscited by 122
  27. 2021Generative Adversarial Transformerscited by 120
  28. 2021CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networkscited by 118
  29. 2021A Survey of Visual Transformerscited by 117
  30. 2021DropGNN: Random Dropouts Increase the Expressiveness of Graph Neural Networkscited by 117
  31. 2021Geometric Deep Learning on Molecular Representationscited by 113
  32. 2021Causal Attention for Vision-Language Taskscited by 111
  33. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  34. 2021SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networkscited by 108
  35. 2021GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensemblescited by 106
  36. 2022FoSR: First-order spectral rewiring for addressing oversquashing in GNNscited by 104
  37. 2022GFlowNets and variational inferencecited by 103
  38. 2023Specformer: Spectral Graph Neural Networks Meet Transformerscited by 102

50 most cited of 1,341 All 1,341 (JSON)