mlsanityData to 7 Oct 2026

Neural Algorithmic Reasoning with Causal Regularisation

arXiv 2302.10258 ↗First posted 20 Feb 2023cs.LG cs.AI stat.MEcited by 30

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent work on neural algorithmic reasoning has investigated the reasoning capabilities of neural networks, effectively demonstrating they can learn to execute classical algorithms on unseen data coming from the train distribution. However, the performance of existing neural reasoners significantly degrades on out-of-distribution (OOD) test data, where inputs have larger sizes. In this work, we make an important observation: there are many different inputs for which an algorithm will perform certain intermediate computations identically. This insight allows us to develop data augmentation procedures that, given an algorithm's intermediate trajectory, produce inputs for which the target algorithm would have exactly the same next trajectory step. We ensure invariance in the next-step prediction across such inputs, by employing a self-supervised objective derived by our observation, formalised in a causal graph. We prove that the resulting method, which we call Hint-ReLIC, improves the OOD generalisation capabilities of the reasoner. We evaluate our method on the CLRS algorithmic reasoning benchmark, where we show up to 3× improvements on the OOD test data.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 43 of its 65 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

43 of 65 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking11 Feb

Other references · 42

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  7. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  8. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  9. 2018Deep Graph Infomaxcited by 1,101
  10. 2014Neural Turing Machinescited by 630
  11. 2018Universal Transformerscited by 613
  12. 2020Deep Graph Contrastive Representation Learningcited by 535
30 more
  1. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  2. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
  3. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  4. 2018Learning a SAT Solver from Single-Bit Supervisioncited by 198
  5. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  6. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 184
  7. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  8. 2019What Can Neural Networks Reason About?cited by 155
  9. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  10. 2020From Local Structures to Size Generalization in Graph Neural Networkscited by 119
  11. 2019Neural Execution of Graph Algorithmscited by 118
  12. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  13. 2020Learning the Travelling Salesperson Problem Requires Rethinking Generalizationcited by 96
  14. 2021Neural Algorithmic Reasoningcited by 92
  15. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  16. 2017Recurrent Relational Networkscited by 71
  17. 2022Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?cited by 67
  18. 2022Theory of Graph Neural Networks: Representation and Learningcited by 60
  19. 2014Visual Causal Feature Learningcited by 59
  20. 2022A Generalist Neural Algorithmic Learnercited by 58
  21. 2022Graph Neural Networks are Dynamic Programmerscited by 55
  22. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  23. 2022OOD Link Prediction Generalization Capabilities of Message-Passing GNNs in Larger Test Graphscited by 38
  24. 2022SizeShiftReg: a Regularization Method for Improving Size-Generalization in Graph Neural Networkscited by 36
  25. 2022Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Taskscited by 29
  26. 2022Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentationscited by 28
  27. 2021Noether Networks: Meta-Learning Useful Conserved Quantitiescited by 26
  28. 2021Neural Algorithmic Reasoners are Implicit Plannerscited by 21
  29. 2021Reasoning-Modulated Representationscited by 13
  30. 2022Learning to Configure Computer Networks with Neural Algorithmic Reasoningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 16; 2025: 5; 2026: 4.

30 papers.

Sort:

By area:

  1. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  2. 2024Simulation of Graph Algorithms with Looped Transformerscited by 29
  3. 2024The CLRS-Text Algorithmic Reasoning Language Benchmarkcited by 28
  4. 2023Neural Algorithmic Reasoning Without Intermediate Supervisioncited by 16
  5. 2023Asynchronous Algorithmic Alignment with Cocyclescited by 13
  6. 2023SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoningcited by 11
  7. 2024On the Markov Property of Neural Algorithmic Reasoning: Analyses and Methodscited by 10
  8. 2023Recursive Algorithmic Reasoningcited by 9
  9. 2024Discrete Neural Algorithmic Reasoningcited by 8
  10. 2024Benchmarking ChatGPT on Algorithmic Reasoningcited by 8
  11. 2025Primal-Dual Neural Algorithmic Reasoningcited by 8
  12. 2023Parallel Algorithms Align with Neural Executioncited by 6
18 more
  1. 2024Recurrent Aggregators in Neural Algorithmic Reasoningcited by 6
  2. 2024Neural Algorithmic Reasoning with Multiple Correct Solutionscited by 4
  3. 2024Deep Equilibrium Algorithmic Reasoningcited by 4
  4. 2025Rethinking Addressing in Language Models via Contexualized Equivariant Positional Encodingcited by 4
  5. 2024NAR-*ICP: Neural Execution of Classical ICP-based Pointcloud Registration Algorithmscited by 2
  6. 2024Open-Book Neural Algorithmic Reasoningcited by 2
  7. 2024The Deep Equilibrium Algorithmic Reasonercited by 1
  8. 2026Algorithmic Task Capture, Computational Complexity, and Inductive Bias of Infinite Transformerscited by 1
  9. 2024A Feature-based Generalizable Prediction Model for Both Perceptual and Abstract Reasoningcited by 0
  10. 2024Spatio-Temporal Graphical Counterfactuals: An Overviewcited by 0
  11. 2024Algorithm-Informed Graph Neural Networks for Leakage Detection and Localization in Water Distribution Networkscited by 0
  12. 2024Discovering Data Structures: Nearest Neighbor Search and Beyondcited by 0
  13. 2025AlgoPilot: Fully Autonomous Program Synthesis Without Human-Written Programscited by 0
  14. 2025KNARsack: Teaching Neural Algorithmic Reasoners to Solve Pseudo-Polynomial Problemscited by 0
  15. 2025Tackling GNARLy Problems: Graph Neural Algorithmic Reasoning Reimagined through Reinforcement Learningcited by 0
  16. 2026Statistical Guarantees for Reasoning Probes on Looped Boolean Circuitscited by 0
  17. 2026Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networkscited by 0
  18. 2026Closed-Loop Graph Algorithm Execution with Small Language Models: Step Accuracy and Rollout Reliabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reasoning Algorithmically in Graph Neural Networks
  2. 2026The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning
  3. 2025TempoBench: Reasoning Execution Without Causal Attribution Is Just Simulation
  4. 2025Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
  5. 2025CausalARC: Abstract Reasoning with Causal World Models
  6. 2026Orca: Neural Operators for Causal Reasoning in Continuous Time
  7. 2026Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models
  8. 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning