mlsanityData to 7 Oct 2026

Heuristic Rank Selection with Progressively Searching Tensor Ring Network

arXiv 2009.10580 ↗First posted 22 Sep 2020cs.CV cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Recently, Tensor Ring Networks (TRNs) have been applied in deep networks, achieving remarkable successes in compression ratio and accuracy. Although highly related to the performance of TRNs, rank selection is seldom studied in previous works and usually set to equal in experiments. Meanwhile, there is not any heuristic method to choose the rank, and an enumerating way to find appropriate rank is extremely time-consuming. Interestingly, we discover that part of the rank elements is sensitive and usually aggregate in a narrow region, namely an interest region. Therefore, based on the above phenomenon, we propose a novel progressive genetic algorithm named Progressively Searching Tensor Ring Network Search (PSTRN), which has the ability to find optimal rank precisely and efficiently. Through the evolutionary phase and progressive phase, PSTRN can converge to the interest region quickly and harvest good performance. Experimental results show that PSTRN can significantly reduce the complexity of seeking rank, compared with the enumerating method. Furthermore, our method is validated on public benchmarks like MNIST, CIFAR10/100, UCF11 and HMDB51, achieving the state-of-the-art performance.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 7; 2024: 8; 2025: 4; 2026: 1.

28 papers.

Sort:

  1. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  2. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29
  3. 2023HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networkscited by 20
  4. 2023Reusing Pretrained Models by Multi-linear Operators for Efficient Trainingcited by 15
  5. 2022Permutation Search of Tensor Network Structures via Local Samplingcited by 13
  6. 2023Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluationscited by 12
  7. 2024ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networkscited by 11
  8. 2022BViT: Broad Attention based Vision Transformercited by 10
  9. 2024Preparing Lessons for Progressive Training on Language Modelscited by 10
  10. 2021TedNet: A Pytorch Toolkit for Tensor Decomposition Networkscited by 9
  11. 2023SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspectivecited by 9
  12. 2022A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networkscited by 8
16 more
  1. 2023Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Trainingcited by 5
  2. 2024tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)cited by 5
  3. 2025IDInit: A Universal and Stable Initialization Method for Neural Network Trainingcited by 5
  4. 2021ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Searchcited by 4
  5. 2022CSTAR: Towards Compact and STructured Deep Neural Networks with Adversarial Robustnesscited by 4
  6. 2021Stacked BNAS: Rethinking Broad Convolutional Neural Network for Neural Architecture Searchcited by 3
  7. 2025Multiscale Tensor Summation Factorization as a New Neural Network Layer (MTS Layer) for Multidimensional Data Processingcited by 2
  8. 2023A Differentiable Framework for End-to-End Learning of Hybrid Structured Compressioncited by 1
  9. 2024An Adaptive Tensor-Train Decomposition Approach for Efficient Deep Neural Network Compressioncited by 1
  10. 2024Convolutional Neural Network Compression via Dynamic Parameter Rank Pruningcited by 0
  11. 2024Reduced storage direct tensor ring decomposition for convolutional neural networks compressioncited by 0
  12. 2024Convolutional Neural Network Compression Based on Low-Rank Decompositioncited by 0
  13. 2024Unified Framework for Pre-trained Neural Network Compression via Decomposition and Optimized Rank Selectioncited by 0
  14. 2025FGFP: A Fractional Gaussian Filter and Pruning for Deep Neural Networks Compressioncited by 0
  15. 2025Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Modelscited by 0
  16. 2026Tensor Decomposition Structure Search Framework from an Interaction Perspectivecited by 0