# tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)

arXiv: https://arxiv.org/abs/2402.02456 · HTML: https://mlsanity.com/paper/2402.02456/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 69 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Builds on (most cited first, 28 of 28)

- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation](https://mlsanity.com/paper/2310.02304/) (2023)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [Neural Optimizer Search with Reinforcement Learning](https://mlsanity.com/paper/1709.07417/) (2017)
- [Language Models Can Teach Themselves to Program Better](https://mlsanity.com/paper/2207.14502/) (2022)
- [Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination](https://mlsanity.com/paper/1401.6497/) (2014)
- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives](https://mlsanity.com/paper/1708.09165/) (2017)
- [Algorithm Evolution Using Large Language Model](https://mlsanity.com/paper/2311.15249/) (2023)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Evolving Code with A Large Language Model](https://mlsanity.com/paper/2401.07102/) (2024)
- [Smooth PARAFAC Decomposition for Tensor Completion](https://mlsanity.com/paper/1505.06611/) (2015)
- [Fully Autonomous Programming with Large Language Models](https://mlsanity.com/paper/2304.10423/) (2023)
- [Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain](https://mlsanity.com/paper/2310.14053/) (2023)
- [Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition](https://mlsanity.com/paper/1710.07324/) (2017)
- [Heuristic Rank Selection with Progressively Searching Tensor Ring Network](https://mlsanity.com/paper/2009.10580/) (2020)
- [Einconv: Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks](https://mlsanity.com/paper/1908.04471/) (2019)
- [Solving high-dimensional parabolic PDEs using the tensor train format](https://mlsanity.com/paper/2102.11830/) (2021)
- [Permutation Search of Tensor Network Structures via Local Sampling](https://mlsanity.com/paper/2206.06597/) (2022)
- [Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations](https://mlsanity.com/paper/2304.12875/) (2023)
- [Adaptive Learning of Tensor Network Structures](https://mlsanity.com/paper/2008.05437/) (2020)
- [Efficient Non-Parametric Optimizer Search for Diverse Tasks](https://mlsanity.com/paper/2209.13575/) (2022)
- [MARS: Masked Automatic Ranks Selection in Tensor Decompositions](https://mlsanity.com/paper/2006.10859/) (2020)

## Built on by (most cited first, 5 of 5)

- [Multi-objective Evolution of Heuristic Using Large Language Model](https://mlsanity.com/paper/2409.16867/) (2024)
- [Domain-Aware Tensor Network Structure Search](https://mlsanity.com/paper/2505.23537/) (2025)
- [Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection](https://mlsanity.com/paper/2410.10728/) (2024)
- [EvoVLMA: Evolutionary Vision-Language Model Adaptation](https://mlsanity.com/paper/2508.01558/) (2025)
- [Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks](https://mlsanity.com/paper/2606.01975/) (2026)

## Abstract (arXiv)

Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods.