Data to 7 Oct 2026

Ultimate tensorization: compressing convolutional and FC layers alike

arXiv 1611.03214 ↗First posted 10 Nov 2016cs.LGcited by 55

Not on the map. Why?

Abstract (arXiv)

Convolutional neural networks excel in image recognition tasks, but this comes at the cost of high computational and memory complexity. To tackle this problem, [1] developed a tensor factorization framework to compress fully-connected layers. In this paper, we focus on compressing convolutional layers. We show that while the direct application of the tensor framework [1] to the 4-dimensional kernel of convolution does compress the layer, we can do better. We reshape the convolutional kernel into a tensor of higher order and factorize it. We combine the proposed approach with the previous work to compress both convolutional and fully-connected layers of a network and achieve 80x network compression rate with 1.1% accuracy drop on the CIFAR-10 dataset.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 13; 2023: 8; 2024: 9; 2025: 5; 2026: 1.

55 papers.

Sort:

By area:

2024Task Singular Vectors: Reducing Task Interference in Model Mergingcited by 143
2021Tensor Methods in Computer Vision and Deep Learningcited by 52
2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
2022Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Modelscited by 31
2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29
2021Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operatorscited by 28
2024Multilinear Mixture of Experts: Scalable Expert Specialization through Factorizationcited by 28
2021QTN-VQC: An End-to-End Learning framework for Quantum Neural Networkscited by 27
2021Convolutional Neural Network Compression through Generalized Kronecker Product Decompositioncited by 14
2023Tensor Decomposition for Model Reduction in Neural Networks: A Reviewcited by 14
2021Joint Matrix Decomposition for Deep Convolutional Neural Networks Compressioncited by 10

38 more

2022Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processingcited by 10
2021Quantum-inspired event reconstruction with Tensor Networks: Matrix Product Statescited by 9
2022Towards Practical Control of Singular Values of Convolutional Layerscited by 9
2022A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networkscited by 8
2021More Efficient Sampling for Tensor Decomposition With Worst-Case Guaranteescited by 7
2022Classical versus Quantum: comparing Tensor Network-based Quantum Circuits on LHC datacited by 7
2022TT-NF: Tensor Train Neural Fieldscited by 7
2023On the Opportunities of Green Computing: A Surveycited by 7
2021Spectral Tensor Train Parameterization of Deep Learning Layerscited by 6
2021Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classificationcited by 6
2022Update Compression for Deep Neural Networks on the Edgecited by 6
2022Nimble GNN Embedding with Tensor-Train Decompositioncited by 6
2025LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossingcited by 6
2023Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Trainingcited by 5
2024Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillationcited by 5
2021LightLayers: Parameter Efficient Dense and Convolutional Layers for Image Classificationcited by 4
2022FAMLP: A Frequency-Aware MLP-Like Architecture For Domain Generalizationcited by 4
2025MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuningcited by 4
20213U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Accelerationcited by 3
2021Tensor Yard: One-Shot Algorithm of Hardware-Friendly Tensor-Train Decomposition for Convolutional Neural Networkscited by 3
2021Semi-tensor Product-based TensorDecomposition for Neural Network Compressioncited by 3
2022Position: Tensor Networks are a Valuable Asset for Green AIcited by 3
2025Transformed Low-rank Adaptation via Tensor Decomposition and Its Applications to Text-to-image Modelscited by 3
2021Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directionscited by 2
2021Low-Rank+Sparse Tensor Compression for Neural Networkscited by 2
2022Deep tensor networks with matrix product operatorscited by 2
2023How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?cited by 2
2025Multiscale Tensor Summation Factorization as a New Neural Network Layer (MTS Layer) for Multidimensional Data Processingcited by 2
2021Exploiting Elasticity in Tensor Ranks for Compressing Neural Networkscited by 1
2022STN: Scalable Tensorizing Networks via Structure-Aware Training and Adaptive Compressioncited by 1
2023On Model Compression for Neural Networks: Framework, Algorithm, and Convergence Guaranteecited by 1
2024conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networkscited by 1
2024An Adaptive Tensor-Train Decomposition Approach for Efficient Deep Neural Network Compressioncited by 1
2021Nonlinear Tensor Ring Networkcited by 0
2022SeKron: A Decomposition Method Supporting Many Factorization Structurescited by 0
2023Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecturecited by 0
2023Enhancing Deep Learning Models through Tensorization: A Comprehensive Survey and Frameworkcited by 0
2024Convolutional Neural Network Compression via Dynamic Parameter Rank Pruningcited by 0

50 most cited of 55 All 55 (JSON)