mlsanityData to 7 Oct 2026

Permutation Search of Tensor Network Structures via Local Sampling

arXiv 2206.06597 ↗First posted 14 Jun 2022cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Recent works put much effort into tensor network structure search (TN-SS), aiming to select suitable tensor network (TN) structures, involving the TN-ranks, formats, and so on, for the decomposition or learning tasks. In this paper, we consider a practical variant of TN-SS, dubbed TN permutation search (TN-PS), in which we search for good mappings from tensor modes onto TN vertices (core tensors) for compact TN representations. We conduct a theoretical investigation of TN-PS and propose a practically-efficient algorithm to resolve the problem. Theoretically, we prove the counting and metric properties of search spaces of TN-PS, analyzing for the first time the impact of TN structures on these unique properties. Numerically, we propose a novel meta-heuristic algorithm, in which the searching is done by randomly sampling in a neighborhood established in our theory, and then recurrently updating the neighborhood until convergence. Numerical results demonstrate that the new algorithm can reduce the required model size of TNs in extensive benchmarks, implying the improvement in the expressive power of TNs. Furthermore, the computational cost for the new algorithm is significantly less than that in~citeli2020evolutionary.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 29 of its 73 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

29 of 73 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Tensorizing Neural Networkscited by 246
  3. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 160
  4. 2016Tensor Ring Decompositioncited by 125
  5. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  6. 2018Towards Quantum Machine Learning with Tensor Networkscited by 74
  7. 2014Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determinationcited by 65
  8. 2016Efficient tensor completion for color image and video recovery: Low-rank tensor traincited by 61
  9. 2019Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learningcited by 55
  10. 2017Efficient Low Rank Tensor Ring Completioncited by 45
  11. 2017Tensor Regression Networkscited by 44
  12. 2021Implicit Regularization in Tensor Factorizationcited by 43
17 more
  1. 2018Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completioncited by 42
  2. 2019Gradientless Descent: High-Dimensional Zeroth-Order Optimizationcited by 42
  3. 2015Smooth PARAFAC Decomposition for Tensor Completioncited by 34
  4. 2021Tensor-Train Density Estimationcited by 30
  5. 2017Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decompositioncited by 28
  6. 2020Heuristic Rank Selection with Progressively Searching Tensor Ring Networkcited by 28
  7. 2022Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networkscited by 27
  8. 2019Bayesian Tensorized Neural Networks with Automatic Rank Selectioncited by 24
  9. 2019Einconv: Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networkscited by 24
  10. 2021Tensor networks for unsupervised machine learningcited by 20
  11. 2021Solving high-dimensional parabolic PDEs using the tensor train formatcited by 17
  12. 2020A Multi-Scale Tensor Network Architecture for Classification and Regressioncited by 13
  13. 2018High-dimension Tensor Completion via Gradient-based Optimization Under Tensor-train Formatcited by 12
  14. 2019Domain-independent Dominance of Adaptive Methodscited by 12
  15. 2020Continuum-Armed Bandits: A Function Space Perspectivecited by 8
  16. 2021On the Memory Mechanism of Tensor-Power Recurrent Modelscited by 5
  17. 2020MARS: Masked Automatic Ranks Selection in Tensor Decompositionscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 4; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023TensorGPT: Efficient Compression of Large Language Models based on Tensor-Train Decompositioncited by 15
  2. 2023Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluationscited by 12
  3. 2023SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspectivecited by 9
  4. 2024tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)cited by 5
  5. 2025Domain-Aware Tensor Network Structure Searchcited by 5
  6. 2024Efficient Nonparametric Tensor Decomposition for Binary and Count Datacited by 3
  7. 2024Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selectioncited by 3
  8. 2023TERM Model: Tensor Ring Mixture Model for Density Estimationcited by 2
  9. 2025Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularizationcited by 1
  10. 2024Reduced storage direct tensor ring decomposition for convolutional neural networks compressioncited by 0
  11. 2025Score-Based Model for Low-Rank Tensor Recoverycited by 0
  12. 2025Renormalization Group Guided Tensor Network Structure Searchcited by 0
1 more
  1. 2026Tensor Decomposition Structure Search Framework from an Interaction Perspectivecited by 0