# TWIG: Towards pre-hoc Hyperparameter Optimisation and Cross-Graph Generalisation via Simulated KGE Models

arXiv: https://arxiv.org/abs/2402.06097 · HTML: https://mlsanity.com/paper/2402.06097/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 8 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings](https://mlsanity.com/paper/2006.13009/) (2020)
- [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018)
- [NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs](https://mlsanity.com/paper/2106.12144/) (2021)
- [Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework](https://mlsanity.com/paper/2006.13365/) (2020)
- [Data Poisoning Attack against Knowledge Graph Embedding](https://mlsanity.com/paper/1904.12052/) (2019)
- [Co-embedding of Nodes and Edges with Graph Neural Networks](https://mlsanity.com/paper/2010.13242/) (2020)
- [OpenBioLink: A benchmarking framework for large-scale biomedical link prediction](https://mlsanity.com/paper/1912.04616/) (2019)
- [Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2103.10367/) (2021)
- [Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1708.06816/) (2017)
- [Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods](https://mlsanity.com/paper/2111.03120/) (2021)
- [Knowledge Base Completion: Baseline strikes back (Again)](https://mlsanity.com/paper/2005.00804/) (2020)
- [Implications of Topological Imbalance for Representation Learning on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2112.06567/) (2021)
- [Explaining Neural Matrix Factorization with Gradient Rollback](https://mlsanity.com/paper/2010.05516/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Survey on Knowledge Graph Structure and Knowledge Graph Embeddings](https://mlsanity.com/paper/2412.10092/) (2024)
- [Extending TWIG: Zero-Shot Predictive Hyperparameter Selection for KGEs based on Graph Structure](https://mlsanity.com/paper/2412.14801/) (2024)

## Abstract (arXiv)

In this paper we introduce TWIG (Topologically-Weighted Intelligence Generation), a novel, embedding-free paradigm for simulating the output of KGEs that uses a tiny fraction of the parameters. TWIG learns weights from inputs that consist of topological features of the graph data, with no coding for latent representations of entities or edges. Our experiments on the UMLS dataset show that a single TWIG neural network can predict the results of state-of-the-art ComplEx-N3 KGE model nearly exactly on across all hyperparameter configurations. To do this it uses a total of 2590 learnable parameters, but accurately predicts the results of 1215 different hyperparameter combinations with a combined cost of 29,322,000 parameters. Based on these results, we make two claims: 1) that KGEs do not learn latent semantics, but only latent representations of structural patterns; 2) that hyperparameter choice in KGEs is a deterministic function of the KGE model and graph structure. We further hypothesise that, as TWIG can simulate KGEs without embeddings, that node and edge embeddings are not needed to learn to accurately predict new facts in KGs. Finally, we formulate all of our findings under the umbrella of the ``Structural Generalisation Hypothesis", which suggests that ``twiggy" embedding-free / data-structure-based learning methods can allow a single neural network to simulate KGE performance, and perhaps solve the Link Prediction task, across many KGs from diverse domains and with different semantics.