# Searching to Sparsify Tensor Decomposition for N-ary Relational Data arXiv: https://arxiv.org/abs/2104.10625 ยท HTML: https://mlsanity.com/paper/2104.10625/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 25 of its 53 references that are in the map. - First posted on arXiv 21 Apr 2021. ## Builds on (most cited first, 25 of 25) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018) - [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014) - [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019) - [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017) - [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017) - [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019) - [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018) - [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015) - [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018) - [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019) - [Canonical Tensor Decomposition for Knowledge Base Completion](https://mlsanity.com/paper/1806.07297/) (2018) - [Tensor Ring Decomposition](https://mlsanity.com/paper/1606.05535/) (2016) - [Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences](https://mlsanity.com/paper/1902.06236/) (2019) - [Knowledge Graph Completion via Complex Tensor Factorization](https://mlsanity.com/paper/1702.06879/) (2017) - [Analogical Inference for Multi-Relational Embeddings](https://mlsanity.com/paper/1705.02426/) (2017) - [Automated Machine Learning: From Principles to Practices](https://mlsanity.com/paper/1810.13306/) (2018) - [On the representation and embedding of knowledge bases beyond binary relations](https://mlsanity.com/paper/1604.08642/) (2016) - [Hypernetwork Knowledge Graph Embeddings](https://mlsanity.com/paper/1808.07018/) (2018) - [Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases](https://mlsanity.com/paper/2007.03988/) (2020) - [AutoSF: Searching Scoring Functions for Knowledge Graph Embedding](https://mlsanity.com/paper/1904.11682/) (2019) - [Efficient Neural Architecture Search via Proximal Iterations](https://mlsanity.com/paper/1905.13577/) (2019) - [Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search](https://mlsanity.com/paper/1905.08537/) (2019) - [On Multi-Relational Link Prediction with Bilinear Models](https://mlsanity.com/paper/1709.04808/) (2017) - [Irreflexive and Hierarchical Relations as Translations](https://mlsanity.com/paper/1304.7158/) (2013) ## Built on by (most cited first, 13 of 13) - [What is Event Knowledge Graph: A Survey](https://mlsanity.com/paper/2112.15280/) (2021) - [AutoGEL: An Automated Graph Neural Network with Explicit Link Information](https://mlsanity.com/paper/2112.01064/) (2021) - [Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers](https://mlsanity.com/paper/2305.18256/) (2023) - [Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks](https://mlsanity.com/paper/2308.06960/) (2023) - [Shrinking Embeddings for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2306.02199/) (2023) - [Temporal Fact Reasoning over Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2307.10219/) (2023) - [A Survey of Link Prediction in N-ary Knowledge Graphs](https://mlsanity.com/paper/2506.08970/) (2025) - [Few-shot Link Prediction on N-ary Facts](https://mlsanity.com/paper/2305.06104/) (2023) - [HyperMono: A Monotonicity-aware Approach to Hyper-Relational Knowledge Representation](https://mlsanity.com/paper/2404.09848/) (2024) - [Generalizing Hyperedge Expansion for Hyper-relational Knowledge Graph Modeling](https://mlsanity.com/paper/2411.06191/) (2024) - [Proficient Graph Neural Network Design by Accumulating Knowledge on Large Language Models](https://mlsanity.com/paper/2408.06717/) (2024) - [Two-dimensional Taxonomy for N-ary Knowledge Representation Learning Methods](https://mlsanity.com/paper/2506.05626/) (2025) - [Generative Representation Learning on Hyper-relational Knowledge Graphs via Masked Discrete Diffusion](https://mlsanity.com/paper/2605.24064/) (2026) ## Abstract (arXiv) Tensor, an extension of the vector and matrix to the multi-dimensional case, is a natural way to describe the N-ary relational data. Recently, tensor decomposition methods have been introduced into N-ary relational data and become state-of-the-art on embedding learning. However, the performance of existing tensor decomposition methods is not as good as desired. First, they suffer from the data-sparsity issue since they can only learn from the N-ary relational data with a specific arity, i.e., parts of common N-ary relational data. Besides, they are neither effective nor efficient enough to be trained due to the over-parameterization problem. In this paper, we propose a novel method, i.e., S2S, for effectively and efficiently learning from the N-ary relational data. Specifically, we propose a new tensor decomposition framework, which allows embedding sharing to learn from facts with mixed arity. Since the core tensors may still suffer from the over-parameterization, we propose to reduce parameters by sparsifying the core tensors while retaining their expressive power using neural architecture search (NAS) techniques, which can search for data-dependent architectures. As a result, the proposed S2S not only guarantees to be expressive but also efficiently learns from mixed arity. Finally, empirical results have demonstrated that S2S is efficient to train and achieves state-of-the-art performance.