# Learning through structure: towards deep neuromorphic knowledge graph embeddings

arXiv: https://arxiv.org/abs/2109.10376 · HTML: https://mlsanity.com/paper/2109.10376/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks](https://mlsanity.com/paper/2102.03322/) (2021)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Towards deep learning with segregated dendrites](https://mlsanity.com/paper/1610.00161/) (2016)
- [Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks](https://mlsanity.com/paper/2009.08378/) (2020)
- [Scene Graph Reasoning for Visual Question Answering](https://mlsanity.com/paper/2007.01072/) (2020)
- [Person Re-identification with Deep Similarity-Guided Graph Neural Network](https://mlsanity.com/paper/1807.09975/) (2018)
- [Sparse Spiking Gradient Descent](https://mlsanity.com/paper/2105.08810/) (2021)
- [Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2103.10367/) (2021)
- [SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron](https://mlsanity.com/paper/1903.02440/) (2019)
- [Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain](https://mlsanity.com/paper/1805.08932/) (2018)
- [Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate](https://mlsanity.com/paper/1912.12980/) (2019)
- [Demonstrating Advantages of Neuromorphic Computation: A Pilot Study](https://mlsanity.com/paper/1811.03618/) (2018)
- [Improving Visual Relationship Detection using Semantic Modeling of Scene Descriptions](https://mlsanity.com/paper/1809.00204/) (2018)
- [SpikE: spike-based embeddings for multi-relational graph data](https://mlsanity.com/paper/2104.13398/) (2021)
- [Machine learning on knowledge graphs for context-aware security monitoring](https://mlsanity.com/paper/2105.08741/) (2021)
- [SemifreddoNets: Partially Frozen Neural Networks for Efficient Computer Vision Systems](https://mlsanity.com/paper/2006.06888/) (2020)

## Built on by (most cited first, 7 of 7)

- [Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks](https://mlsanity.com/paper/2208.10364/) (2022)
- [R-GCN: The R Could Stand for Random](https://mlsanity.com/paper/2203.02424/) (2022)
- [On-device Online Learning and Semantic Management of TinyML Systems](https://mlsanity.com/paper/2405.07601/) (2024)
- [Relational representation learning with spike trains](https://mlsanity.com/paper/2205.09140/) (2022)
- [SpikeSTAG: Spatial-Temporal Forecasting via GNN-SNN Collaboration](https://mlsanity.com/paper/2508.02069/) (2025)
- [Neuro-symbolic computing with spiking neural networks](https://mlsanity.com/paper/2208.02576/) (2022)
- [ChronoSpike: An Adaptive Spiking Graph Neural Network for Dynamic Graphs](https://mlsanity.com/paper/2602.01124/) (2026)

## Abstract (arXiv)

Computing latent representations for graph-structured data is an ubiquitous learning task in many industrial and academic applications ranging from molecule synthetization to social network analysis and recommender systems. Knowledge graphs are among the most popular and widely used data representations related to the Semantic Web. Next to structuring factual knowledge in a machine-readable format, knowledge graphs serve as the backbone of many artificial intelligence applications and allow the ingestion of context information into various learning algorithms. Graph neural networks attempt to encode graph structures in low-dimensional vector spaces via a message passing heuristic between neighboring nodes. Over the recent years, a multitude of different graph neural network architectures demonstrated ground-breaking performances in many learning tasks. In this work, we propose a strategy to map deep graph learning architectures for knowledge graph reasoning to neuromorphic architectures. Based on the insight that randomly initialized and untrained (i.e., frozen) graph neural networks are able to preserve local graph structures, we compose a frozen neural network with shallow knowledge graph embedding models. We experimentally show that already on conventional computing hardware, this leads to a significant speedup and memory reduction while maintaining a competitive performance level. Moreover, we extend the frozen architecture to spiking neural networks, introducing a novel, event-based and highly sparse knowledge graph embedding algorithm that is suitable for implementation in neuromorphic hardware.