# Permutation Search of Tensor Network Structures via Local Sampling

arXiv: https://arxiv.org/abs/2206.06597 · HTML: https://mlsanity.com/paper/2206.06597/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 73 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning](https://mlsanity.com/paper/2006.06224/) (2020)
- [Tensor Ring Decomposition](https://mlsanity.com/paper/1606.05535/) (2016)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [Towards Quantum Machine Learning with Tensor Networks](https://mlsanity.com/paper/1803.11537/) (2018)
- [Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination](https://mlsanity.com/paper/1401.6497/) (2014)
- [Efficient tensor completion for color image and video recovery: Low-rank tensor train](https://mlsanity.com/paper/1606.01500/) (2016)
- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [Efficient Low Rank Tensor Ring Completion](https://mlsanity.com/paper/1707.08184/) (2017)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Implicit Regularization in Tensor Factorization](https://mlsanity.com/paper/2102.09972/) (2021)
- [Gradientless Descent: High-Dimensional Zeroth-Order Optimization](https://mlsanity.com/paper/1911.06317/) (2019)
- [Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion](https://mlsanity.com/paper/1809.02288/) (2018)
- [Smooth PARAFAC Decomposition for Tensor Completion](https://mlsanity.com/paper/1505.06611/) (2015)
- [Tensor-Train Density Estimation](https://mlsanity.com/paper/2108.00089/) (2021)
- [Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition](https://mlsanity.com/paper/1710.07324/) (2017)
- [Heuristic Rank Selection with Progressively Searching Tensor Ring Network](https://mlsanity.com/paper/2009.10580/) (2020)
- [Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks](https://mlsanity.com/paper/2201.11729/) (2022)
- [Bayesian Tensorized Neural Networks with Automatic Rank Selection](https://mlsanity.com/paper/1905.10478/) (2019)
- [Einconv: Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks](https://mlsanity.com/paper/1908.04471/) (2019)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Solving high-dimensional parabolic PDEs using the tensor train format](https://mlsanity.com/paper/2102.11830/) (2021)
- [A Multi-Scale Tensor Network Architecture for Classification and Regression](https://mlsanity.com/paper/2001.08286/) (2020)
- [Domain-independent Dominance of Adaptive Methods](https://mlsanity.com/paper/1912.01823/) (2019)
- [High-dimension Tensor Completion via Gradient-based Optimization Under Tensor-train Format](https://mlsanity.com/paper/1804.01983/) (2018)
- [Continuum-Armed Bandits: A Function Space Perspective](https://mlsanity.com/paper/2010.08007/) (2020)
- [On the Memory Mechanism of Tensor-Power Recurrent Models](https://mlsanity.com/paper/2103.01521/) (2021)
- [MARS: Masked Automatic Ranks Selection in Tensor Decompositions](https://mlsanity.com/paper/2006.10859/) (2020)

## Built on by (most cited first, 13 of 13)

- [TensorGPT: Efficient Compression of Large Language Models based on Tensor-Train Decomposition](https://mlsanity.com/paper/2307.00526/) (2023)
- [Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations](https://mlsanity.com/paper/2304.12875/) (2023)
- [SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective](https://mlsanity.com/paper/2305.14912/) (2023)
- [tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)](https://mlsanity.com/paper/2402.02456/) (2024)
- [Domain-Aware Tensor Network Structure Search](https://mlsanity.com/paper/2505.23537/) (2025)
- [Efficient Nonparametric Tensor Decomposition for Binary and Count Data](https://mlsanity.com/paper/2401.07711/) (2024)
- [Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection](https://mlsanity.com/paper/2410.10728/) (2024)
- [TERM Model: Tensor Ring Mixture Model for Density Estimation](https://mlsanity.com/paper/2312.08075/) (2023)
- [Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization](https://mlsanity.com/paper/2506.22134/) (2025)
- [Reduced storage direct tensor ring decomposition for convolutional neural networks compression](https://mlsanity.com/paper/2405.10802/) (2024)
- [Score-Based Model for Low-Rank Tensor Recovery](https://mlsanity.com/paper/2506.22295/) (2025)
- [Renormalization Group Guided Tensor Network Structure Search](https://mlsanity.com/paper/2512.24663/) (2025)
- [Tensor Decomposition Structure Search Framework from an Interaction Perspective](https://mlsanity.com/paper/2603.02720/) (2026)

## Abstract (arXiv)

Recent works put much effort into tensor network structure search (TN-SS), aiming to select suitable tensor network (TN) structures, involving the TN-ranks, formats, and so on, for the decomposition or learning tasks. In this paper, we consider a practical variant of TN-SS, dubbed TN permutation search (TN-PS), in which we search for good mappings from tensor modes onto TN vertices (core tensors) for compact TN representations. We conduct a theoretical investigation of TN-PS and propose a practically-efficient algorithm to resolve the problem. Theoretically, we prove the counting and metric properties of search spaces of TN-PS, analyzing for the first time the impact of TN structures on these unique properties. Numerically, we propose a novel meta-heuristic algorithm, in which the searching is done by randomly sampling in a neighborhood established in our theory, and then recurrently updating the neighborhood until convergence. Numerical results demonstrate that the new algorithm can reduce the required model size of TNs in extensive benchmarks, implying the improvement in the expressive power of TNs. Furthermore, the computational cost for the new algorithm is significantly less than that in~citeli2020evolutionary.