# SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective

arXiv: https://arxiv.org/abs/2305.14912 · HTML: https://mlsanity.com/paper/2305.14912/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 16 of 16)

- [Tensor Ring Decomposition](https://mlsanity.com/paper/1606.05535/) (2016)
- [Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination](https://mlsanity.com/paper/1401.6497/) (2014)
- [Efficient tensor completion for color image and video recovery: Low-rank tensor train](https://mlsanity.com/paper/1606.01500/) (2016)
- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion](https://mlsanity.com/paper/1809.02288/) (2018)
- [Heuristic Rank Selection with Progressively Searching Tensor Ring Network](https://mlsanity.com/paper/2009.10580/) (2020)
- [Tensor N-tubal rank and its convex relaxation for low-rank tensor recovery](https://mlsanity.com/paper/1812.00688/) (2018)
- [Multiscale Tensor Decomposition and Rendering Equation Encoding for View Synthesis](https://mlsanity.com/paper/2303.03808/) (2023)
- [Permutation Search of Tensor Network Structures via Local Sampling](https://mlsanity.com/paper/2206.06597/) (2022)
- [High-dimension Tensor Completion via Gradient-based Optimization Under Tensor-train Format](https://mlsanity.com/paper/1804.01983/) (2018)
- [Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations](https://mlsanity.com/paper/2304.12875/) (2023)
- [Moving Object Detection under Discontinuous Change in Illumination Using Tensor Low-Rank and Invariant Sparse Decomposition](https://mlsanity.com/paper/1904.03175/) (2019)
- [Approximately Optimal Core Shapes for Tensor Decompositions](https://mlsanity.com/paper/2302.03886/) (2023)
- [Adaptive Learning of Tensor Network Structures](https://mlsanity.com/paper/2008.05437/) (2020)
- [Nonlocal Low-Rank Tensor Factor Analysis for Image Restoration](https://mlsanity.com/paper/1803.06795/) (2018)
- [Adaptively Topological Tensor Network for Multi-view Subspace Clustering](https://mlsanity.com/paper/2305.00716/) (2023)

## Built on by (most cited first, 9 of 9)

- [CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization](https://mlsanity.com/paper/2405.14377/) (2024)
- [TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs](https://mlsanity.com/paper/2501.15674/) (2025)
- [Domain-Aware Tensor Network Structure Search](https://mlsanity.com/paper/2505.23537/) (2025)
- [Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection](https://mlsanity.com/paper/2410.10728/) (2024)
- [Haar Nuclear Norms with Applications to Remote Sensing Imagery Restoration](https://mlsanity.com/paper/2407.08509/) (2024)
- [Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization](https://mlsanity.com/paper/2506.22134/) (2025)
- [Renormalization Group Guided Tensor Network Structure Search](https://mlsanity.com/paper/2512.24663/) (2025)
- [Tensor Decomposition Structure Search Framework from an Interaction Perspective](https://mlsanity.com/paper/2603.02720/) (2026)
- [Hierarchical rank-evolving representation for physics-informed neural networks](https://mlsanity.com/paper/2608.09483/) (2026)

## Abstract (arXiv)

Tensor network (TN) representation is a powerful technique for computer vision and machine learning. TN structure search (TN-SS) aims to search for a customized structure to achieve a compact representation, which is a challenging NP-hard problem. Recent "sampling-evaluation"-based methods require sampling an extensive collection of structures and evaluating them one by one, resulting in prohibitively high computational costs. To address this issue, we propose a novel TN paradigm, named SVD-inspired TN decomposition (SVDinsTN), which allows us to efficiently solve the TN-SS problem from a regularized modeling perspective, eliminating the repeated structure evaluations. To be specific, by inserting a diagonal factor for each edge of the fully-connected TN, SVDinsTN allows us to calculate TN cores and diagonal factors simultaneously, with the factor sparsity revealing a compact TN structure. In theory, we prove a convergence guarantee for the proposed method. Experimental results demonstrate that the proposed method achieves approximately 100 to 1000 times acceleration compared to the state-of-the-art TN-SS methods while maintaining a comparable level of representation ability.