mlsanityData to 7 Oct 2026

Persistent Message Passing

arXiv 2103.01043 ↗First posted 1 Mar 2021cs.LG cs.AI cs.SIcited by 13

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) are a powerful inductive bias for modelling algorithmic reasoning procedures and data structures. Their prowess was mainly demonstrated on tasks featuring Markovian dynamics, where querying any associated data structure depends only on its latest state. For many tasks of interest, however, it may be highly beneficial to support efficient data structure queries dependent on previous states. This requires tracking the data structure's evolution through time, placing significant pressure on the GNN's latent representations. We introduce Persistent Message Passing (PMP), a mechanism which endows GNNs with capability of querying past state by explicitly persisting it: rather than overwriting node representations, it creates new nodes whenever required. PMP generalises out-of-distribution to more than 2x larger test inputs on dynamic temporal range queries, significantly outperforming GNNs which overwrite states.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 1 Mar 2021.

Builds on

18 of 25 references.

References · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  3. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  4. 2020Directional Message Passing for Molecular Graphscited by 556
  5. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
  6. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267
  7. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
  8. 2020Solving Mixed Integer Programs Using Neural Networkscited by 178
  9. 2019What Can Neural Networks Reason About?cited by 155
  10. 2017Neural Episodic Controlcited by 148
  11. 2021How to represent part-whole hierarchies in a neural networkcited by 123
  12. 2019Neural Execution of Graph Algorithmscited by 118
6 more
  1. 2020PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterestcited by 75
  2. 2020Pointer Graph Networkscited by 52
  3. 2018Relational inductive bias for physical construction in humans and machinescited by 41
  4. 2020Neural Execution Engines: Learning to Execute Subroutinescited by 36
  5. 2020XLVIN: eXecuted Latent Value Iteration Netscited by 17
  6. 2020Towards Scale-Invariant Graph-related Problem Solving by Iterative Homogeneous Graph Neural Networkscited by 6

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 4; 2023: 4; 2024: 0; 2025: 1.

13 papers.

Sort:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  2. 2022Graph neural networks for materials science and chemistrycited by 181
  3. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  4. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
  5. 2022Graph Neural Networks are Dynamic Programmerscited by 55
  6. 2021Relating Graph Neural Networks to Structural Causal Modelscited by 39
  7. 2023Neural Algorithmic Reasoning for Combinatorial Optimisationcited by 19
  8. 2021Reasoning-Modulated Representationscited by 13
  9. 2023Asynchronous Algorithmic Alignment with Cocyclescited by 13
  10. 2021Algorithmic Concept-based Explainable Reasoningcited by 12
  11. 2023Neural Priority Queues for Graph Neural Networkscited by 7
  12. 2022Memory-Augmented Graph Neural Networks: A Brain-Inspired Reviewcited by 2
1 more
  1. 2025KNARsack: Teaching Neural Algorithmic Reasoners to Solve Pseudo-Polynomial Problemscited by 0