mlsanityData to 13 Oct 2026

Relational recurrent neural networks

arXiv 1806.01822 ↗First posted 5 Jun 2018cs.LG stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to perform complex relational reasoning with the information they remember. Here, we first confirm our intuitions that standard memory architectures may struggle at tasks that heavily involve an understanding of the ways in which entities are connected -- i.e., tasks involving relational reasoning. We then improve upon these deficits by using a new memory module -- a textitRelational Memory Core (RMC) -- which employs multi-head dot product attention to allow memories to interact. Finally, we test the RMC on a suite of tasks that may profit from more capable relational reasoning across sequential information, and show large gains in RL domains (e.g. Mini PacMan), program evaluation, and language modeling, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.

Key facts. Cited by 65 papers in the map (data through 13 Oct 2026). First posted on arXiv 5 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 22; 2022: 17; 2023: 15; 2024: 6; 2025: 2; 2026: 3.

65 papers.

Sort:

By area:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,147
  2. 2021Perceiver: General Perception with Iterative Attentioncited by 1,094
  3. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 625
  4. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267
  5. 2021Conditional Object-Centric Learning from Videocited by 247
  6. 2022Cross-modal Memory Networks for Radiology Report Generationcited by 184
  7. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 123
  8. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 84
  9. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  10. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  11. 2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
  12. 2022Is a Modular Architecture Enough?cited by 36
38 more
  1. 2022Compositional Scene Representation Learning via Reconstruction: A Surveycited by 27
  2. 2022On Neural Architecture Inductive Biases for Relational Taskscited by 26
  3. 2023When can transformers reason with abstract symbols?cited by 23
  4. 2021Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Eventscited by 22
  5. 2021How to transfer algorithmic reasoning knowledge to learn new algorithms?cited by 22
  6. 2021Creativity and Machine Learning: A Surveycited by 21
  7. 2023Systematic Visual Reasoning through Object-Centric Relational Abstractioncited by 20
  8. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  9. 2022Neural Approximation of Graph Topological Featurescited by 16
  10. 2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
  11. 2023Continual Multimodal Knowledge Graph Constructioncited by 15
  12. 2021Reasoning-Modulated Representationscited by 13
  13. 2022Learning to Generalize with Object-centric Agents in the Open World Survival Game Craftercited by 13
  14. 2026Evaluating Memory Structure in LLM Agentscited by 12
  15. 2022Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomescited by 11
  16. 2025SRMT: Shared Memory for Multi-agent Lifelong Pathfindingcited by 8
  17. 2023Composing Task Knowledge with Modular Successor Feature Approximatorscited by 7
  18. 2023Improved Training of Mixture-of-Experts Language GANscited by 7
  19. 2023Concept-Centric Transformers: Enhancing Model Interpretability through Object-Centric Concept Learning within a Shared Global Workspacecited by 7
  20. 2021Learning to Rehearse in Long Sequence Memorizationcited by 6
  21. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5
  22. 2022Learning to Reason With Relational Abstractionscited by 5
  23. 2023Learning Language Representations with Logical Inductive Biascited by 5
  24. 2024Approximation of relation functions and attention mechanismscited by 5
  25. 2024Disentangling and Integrating Relational and Sensory Information in Transformer Architecturescited by 5
  26. 2021Efficient and robust multi-task learning in the brain with modular latent primitivescited by 4
  27. 2023Reusable Slotwise Mechanismscited by 4
  28. 2021Adversarial Machine Learning in Text Analysis and Generationcited by 2
  29. 2021Deep Algorithmic Question Answering: Towards a Compositionally Hybrid AI for Algorithmic Reasoningcited by 2
  30. 2022Semantic Novelty Detection via Relational Reasoningcited by 2
  31. 2022Memory-Augmented Graph Neural Networks: A Brain-Inspired Reviewcited by 2
  32. 2023A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Informationcited by 2
  33. 2023Learning Hierarchical Relational Representations through Relational Convolutionscited by 2
  34. 2023A Framework for Inference Inspired by Human Memory Mechanismscited by 2
  35. 2021State-Space Constraints Can Improve the Generalisation of the Differentiable Neural Computer to Input Sequences With Unseen Lengthcited by 1
  36. 2022Dynamic and Context-Dependent Stock Price Prediction Using Attention Modules and News Sentimentcited by 1
  37. 2022Solving Reasoning Tasks with a Slot Transformercited by 1
  38. 2022Fusion-based Few-Shot Morphing Attack Detection and Fingerprintingcited by 1

50 most cited of 65 All 65 (JSON)