# Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations

arXiv: https://arxiv.org/abs/2304.12875 · HTML: https://mlsanity.com/paper/2304.12875/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 75 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Tensor Ring Decomposition](https://mlsanity.com/paper/1606.05535/) (2016)
- [Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination](https://mlsanity.com/paper/1401.6497/) (2014)
- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives](https://mlsanity.com/paper/1708.09165/) (2017)
- [Efficient Low Rank Tensor Ring Completion](https://mlsanity.com/paper/1707.08184/) (2017)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Gradientless Descent: High-Dimensional Zeroth-Order Optimization](https://mlsanity.com/paper/1911.06317/) (2019)
- [Smooth PARAFAC Decomposition for Tensor Completion](https://mlsanity.com/paper/1505.06611/) (2015)
- [Quantum-inspired Machine Learning on high-energy physics data](https://mlsanity.com/paper/2004.13747/) (2020)
- [Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition](https://mlsanity.com/paper/1710.07324/) (2017)
- [Heuristic Rank Selection with Progressively Searching Tensor Ring Network](https://mlsanity.com/paper/2009.10580/) (2020)
- [Bayesian Tensorized Neural Networks with Automatic Rank Selection](https://mlsanity.com/paper/1905.10478/) (2019)
- [Einconv: Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks](https://mlsanity.com/paper/1908.04471/) (2019)
- [Solving high-dimensional parabolic PDEs using the tensor train format](https://mlsanity.com/paper/2102.11830/) (2021)
- [TTOpt: A Maximum Volume Quantized Tensor Train-based Optimization and its Application to Reinforcement Learning](https://mlsanity.com/paper/2205.00293/) (2022)
- [Permutation Search of Tensor Network Structures via Local Sampling](https://mlsanity.com/paper/2206.06597/) (2022)
- [Approximately Optimal Core Shapes for Tensor Decompositions](https://mlsanity.com/paper/2302.03886/) (2023)
- [More Efficient Sampling for Tensor Decomposition With Worst-Case Guarantees](https://mlsanity.com/paper/2110.07631/) (2021)
- [MARS: Masked Automatic Ranks Selection in Tensor Decompositions](https://mlsanity.com/paper/2006.10859/) (2020)
- [Are Quantum Computers Practical Yet? A Case for Feature Selection in Recommender Systems using Tensor Networks](https://mlsanity.com/paper/2205.04490/) (2022)
- [Tensor Train for Global Optimization Problems in Robotics](https://mlsanity.com/paper/2206.05077/) (2022)

## Built on by (most cited first, 12 of 12)

- [TensorGPT: Efficient Compression of Large Language Models based on Tensor-Train Decomposition](https://mlsanity.com/paper/2307.00526/) (2023)
- [SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective](https://mlsanity.com/paper/2305.14912/) (2023)
- [tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)](https://mlsanity.com/paper/2402.02456/) (2024)
- [Domain-Aware Tensor Network Structure Search](https://mlsanity.com/paper/2505.23537/) (2025)
- [Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection](https://mlsanity.com/paper/2410.10728/) (2024)
- [Understanding the Rank of Tensor Networks via an Intuitive Example-Driven Approach](https://mlsanity.com/paper/2507.10170/) (2025)
- [TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2509.03234/) (2025)
- [Broadcast Product: Redefining Shape-aligned Element-wise Multiplication and Beyond](https://mlsanity.com/paper/2409.17502/) (2024)
- [A quatum inspired neural network for geometric modeling](https://mlsanity.com/paper/2401.01801/) (2024)
- [Reduced storage direct tensor ring decomposition for convolutional neural networks compression](https://mlsanity.com/paper/2405.10802/) (2024)
- [Renormalization Group Guided Tensor Network Structure Search](https://mlsanity.com/paper/2512.24663/) (2025)
- [Tensor Decomposition Structure Search Framework from an Interaction Perspective](https://mlsanity.com/paper/2603.02720/) (2026)

## Abstract (arXiv)

Tensor network (TN) is a powerful framework in machine learning, but selecting a good TN model, known as TN structure search (TN-SS), is a challenging and computationally intensive task. The recent approach TNLS~citeli2022permutation showed promising results for this task, however, its computational efficiency is still unaffordable, requiring too many evaluations of the objective function. We propose TnALE, a new algorithm that updates each structure-related variable alternately by local enumeration, greatly reducing the number of evaluations compared to TNLS. We theoretically investigate the descent steps for TNLS and TnALE, proving that both algorithms can achieve linear convergence up to a constant if a sufficient reduction of the objective is reached in each neighborhood. We also compare the evaluation efficiency of TNLS and TnALE, revealing that Omega(2N) evaluations are typically required in TNLS for reaching the objective reduction in the neighborhood, while ideally O(N2R) evaluations are sufficient in TnALE, where N denotes the tensor order and R reflects the ``low-rankness'' of the neighborhood. Experimental results verify that TnALE can find practically good TN-ranks and permutations with vastly fewer evaluations than the state-of-the-art algorithms.