mlsanityData to 14 Oct 2026

Tensorizing Neural Networks

arXiv 1509.06569 ↗First posted 22 Sep 2015cs.LG cs.NEcited by 250

Not on the map. Why?

Abstract (arXiv)

Deep neural networks currently demonstrate state-of-the-art performance in several domains. At the same time, models of this class are very demanding in terms of computational resources. In particular, a large amount of memory is required by commonly used fully-connected layers, making it hard to use the models on low-end devices and stopping the further increase of the model size. In this paper we convert the dense weight matrices of the fully-connected layers to the Tensor Train format such that the number of parameters is reduced by a huge factor and at the same time the expressive power of the layer is preserved. In particular, for the Very Deep VGG networks we report the compression factor of the dense weight matrix of a fully-connected layer up to 200000 times leading to the compression factor of the whole network up to 7 times.

Key facts. Cited by 250 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Sep 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 56; 2022: 35; 2023: 40; 2024: 37; 2025: 39; 2026: 43.

250 papers.

Sort:

By area:

  1. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 339
  2. 2023Scalable Transformer for PDE Surrogate Modelingcited by 128
  3. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 84
  4. 2023Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEscited by 68
  5. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  6. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  7. 2021Tensor Methods in Computer Vision and Deep Learningcited by 52
  8. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  9. 2021An end-to-end trainable hybrid classical-quantum classifiercited by 51
  10. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
  11. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 32
  12. 2024CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networkscited by 32
38 more
  1. 2022Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Modelscited by 31
  2. 2021Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operatorscited by 29
  3. 2024Compute Better Spent: Replacing Dense Layers with Structured Matricescited by 28
  4. 2025CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activationcited by 28
  5. 2021QTN-VQC: An End-to-End Learning framework for Quantum Neural Networkscited by 27
  6. 2022Theoretical Error Performance Analysis for Variational Quantum Circuit Based Functional Regressioncited by 26
  7. 2024AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuningcited by 25
  8. 2021Spatio-Temporal Pruning and Quantization for Low-latency Spiking Neural Networkscited by 21
  9. 2023HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networkscited by 20
  10. 2021Double Similarity Distillation for Semantic Image Segmentationcited by 19
  11. 2021Local approximate Gaussian process regression for data-driven constitutive laws: Development and comparison with neural networkscited by 17
  12. 2023Tensor networks for quantum machine learningcited by 17
  13. 2022Exploring Extreme Parameter Compression for Pre-trained Language Modelscited by 15
  14. 2024CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimizationcited by 15
  15. 2021Layer Folding: Neural Network Depth Reduction using Activation Linearizationcited by 14
  16. 2021Large-Scale Learning with Fourier Features and Tensor Decompositionscited by 14
  17. 2023Tensor Decomposition for Model Reduction in Neural Networks: A Reviewcited by 14
  18. 2024Tensor Train Low-rank Approximation (TT-LoRA): Democratizing AI with Accelerated LLMscited by 14
  19. 2021Tensor networks and efficient descriptions of classical datacited by 13
  20. 2022Permutation Search of Tensor Network Structures via Local Samplingcited by 13
  21. 2022Federated Learning of Large Models at the Edge via Principal Sub-Model Trainingcited by 13
  22. 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
  23. 2023Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluationscited by 12
  24. 2023Geometry-aware training of factorized layers in tensor Tucker formatcited by 12
  25. 2024LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Modelscited by 12
  26. 2024Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Modelscited by 12
  27. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  28. 2024ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networkscited by 11
  29. 2024Group and Shuffle: Efficient Structured Orthogonal Parametrizationcited by 11
  30. 2025Quantum-PEFT: Ultra parameter-efficient fine-tuningcited by 11
  31. 2026KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matricescited by 11
  32. 2021Joint Matrix Decomposition for Deep Convolutional Neural Networks Compressioncited by 10
  33. 2022Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processingcited by 10
  34. 2023Graph Tensor Networks: An Intuitive Framework for Designing Large-Scale Neural Learning Systems on Multiple Domainscited by 10
  35. 2024Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruningcited by 10
  36. 2021Quantum-inspired event reconstruction with Tensor Networks: Matrix Product Statescited by 9
  37. 2022Quantum-Inspired Tensor Neural Networks for Partial Differential Equationscited by 9
  38. 2022Towards Practical Control of Singular Values of Convolutional Layerscited by 9

50 most cited of 250 All 250 (JSON)