mlsanityData to 7 Oct 2026

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

arXiv 2104.13478 ↗First posted 27 Apr 2021cs.LG cs.AI cs.CGcited by 1,131

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

Key facts. Cited by 1,131 papers in the map (data through 7 Oct 2026). Builds on 117 of its 340 references that are in the map. First posted on arXiv 27 Apr 2021.

Builds on

117 of 340 references.

References · 117

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2016Layer Normalizationcited by 5,579
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  12. 2018How Powerful are Graph Neural Networks?cited by 4,355
38 more
  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  8. 2016Variational Graph Auto-Encoderscited by 1,373
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  10. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  11. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  12. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  13. 2021E(n) Equivariant Graph Neural Networkscited by 951
  14. 2015Pointer Networkscited by 882
  15. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  16. 2015Gated Graph Sequence Neural Networkscited by 792
  17. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  18. 2020A Generalization of Transformer Networks to Graphscited by 653
  19. 2014Neural Turing Machinescited by 630
  20. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  21. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  22. 2020Directional Message Passing for Molecular Graphscited by 556
  23. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  24. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  25. 2020Deep Graph Contrastive Representation Learningcited by 535
  26. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
  27. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  28. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  29. 2020Inductive Representation Learning on Temporal Graphscited by 454
  30. 2018Spherical CNNscited by 452
  31. 2020Lagrangian Neural Networkscited by 388
  32. 2019Provably Powerful Graph Networkscited by 385
  33. 2018How Does Batch Normalization Help Optimization?cited by 355
  34. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  35. 2018Invariant and Equivariant Graph Networkscited by 339
  36. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  37. 2020Improving Graph Neural Network Expressivity via Subgraph Isomorphism Countingcited by 312
  38. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267

50 most cited of 117 All 117 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 73; 2022: 210; 2023: 227; 2024: 195; 2025: 209; 2026: 217.

1,131 papers.

Sort:

By area:

  1. 2021How Attentive are Graph Attention Networks?cited by 818
  2. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  3. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  4. 2022Multimodal Learning with Transformers: A Surveycited by 399
  5. 2022Transformers Learn Shortcuts to Automatacited by 260
  6. 2022Long Range Graph Benchmarkcited by 258
  7. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  8. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 225
  9. 2022Structure-based Drug Design with Equivariant Diffusion Modelscited by 213
  10. 2023Challenges and Opportunities in Quantum Machine Learningcited by 202
  11. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 190
  12. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
38 more
  1. 2021Equivariant Subgraph Aggregation Networkscited by 154
  2. 2023Graph Inductive Biases in Transformers without Message Passingcited by 150
  3. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  4. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  5. 2022Graph-Coupled Oscillator Networkscited by 123
  6. 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 121
  7. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  8. 2021Geometric Deep Learning on Molecular Representationscited by 113
  9. 2023Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNscited by 113
  10. 2022Not too little, not too much: a theoretical analysis of graph (over)smoothingcited by 106
  11. 2022Equivariance with Learned Canonicalization Functionscited by 106
  12. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  13. 2022The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentialscited by 100
  14. 2022Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvaturecited by 96
  15. 2022Approximately Equivariant Networks for Imperfectly Symmetric Dynamicscited by 95
  16. 2022Clifford Neural Layers for PDE Modelingcited by 91
  17. 2023On the Expressive Power of Geometric Graph Neural Networkscited by 89
  18. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  19. 2023Everything is Connected: Graph Neural Networkscited by 85
  20. 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
  21. 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
  22. 2022Lie Point Symmetry Data Augmentation for Neural PDE Solverscited by 76
  23. 2022Expander Graph Propagationcited by 75
  24. 2023A Mathematical Guide to Operator Learningcited by 74
  25. 2022ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphscited by 69
  26. 2023Permutation Equivariant Neural Functionalscited by 69
  27. 2021Equivariant Contrastive Learningcited by 65
  28. 2023A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systemscited by 65
  29. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 65
  30. 2021Geometric Deep Learning and Equivariant Neural Networkscited by 64
  31. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  32. 2022Verifying the Union of Manifolds Hypothesis for Image Datacited by 62
  33. 2022Theory for Equivariant Quantum Neural Networkscited by 60
  34. 2022A Survey of Non-Rigid 3D Registrationcited by 59
  35. 2024Graph Neural Networks for Learning Equivariant Representations of Neural Networkscited by 59
  36. 2022Understanding convolution on graphs via energiescited by 58
  37. 2022Group-Invariant Quantum Machine Learningcited by 56
  38. 2022Geometrically Equivariant Graph Neural Networks: A Surveycited by 55

50 most cited of 1,131 All 1,131 (JSON)