mlsanityData to 14 Oct 2026

Bayesian Tensorized Neural Networks with Automatic Rank Selection

arXiv 1905.10478 ↗First posted 24 May 2019cs.LG stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to achieve this goal, this paper proposes a Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. We provide experimental validation on a fully connected neural network, a CNN and a residual neural network where our work produces 7.4× to 137× more compact neural networks directly from the training.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 7; 2024: 2; 2025: 5; 2026: 2.

26 papers.

Sort:

  1. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  2. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 32
  3. 2023Tensor networks for interpretable and efficient quantum-inspired machine learningcited by 17
  4. 2024CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimizationcited by 15
  5. 2022Permutation Search of Tensor Network Structures via Local Samplingcited by 13
  6. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  7. 2023Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluationscited by 12
  8. 2023Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networkscited by 12
  9. 2024LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Modelscited by 12
  10. 2022TT-PINN: A Tensor-Compressed Neural PDE Solver for Edge Computingcited by 7
  11. 2023Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Trainingcited by 5
  12. 2021Block-Term Tensor Decomposition Model Selection and Computation: The Bayesian Waycited by 4
14 more
  1. 2025Tensorization is a powerful but underexplored tool for compression and interpretability of neural networkscited by 4
  2. 20213U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Accelerationcited by 3
  3. 2021Tensor Yard: One-Shot Algorithm of Hardware-Friendly Tensor-Train Decomposition for Convolutional Neural Networkscited by 3
  4. 2025Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimizationcited by 3
  5. 2021Low-Rank+Sparse Tensor Compression for Neural Networkscited by 2
  6. 2023Quantization-Aware and Tensor-Compressed Training of Transformers for Natural Language Understandingcited by 2
  7. 2025Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networkscited by 2
  8. 2025Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Modelscited by 2
  9. 2022Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Surveycited by 1
  10. 2023Tensorized Hypergraph Neural Networkscited by 1
  11. 2026Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretabilitycited by 1
  12. 2022Tensor Shape Search for Optimum Data Compressioncited by 0
  13. 2025Comprehensive Design Space Exploration for Tensorized Neural Network Hardware Acceleratorscited by 0
  14. 2026Gaussian Universality and Its Breakdown in Tensor-Network Machine Learningcited by 0