mlsanityData to 7 Oct 2026

Compressing Recurrent Neural Network with Tensor Train

arXiv 1705.08052 ↗First posted 23 May 2017cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Recurrent Neural Network (RNN) are a popular choice for modeling temporal and sequential tasks and achieve many state-of-the-art performance on various complex problems. However, most of the state-of-the-art RNNs have millions of parameters and require many computational resources for training and predicting new data. This paper proposes an alternative RNN model to reduce the number of parameters significantly by representing the weight parameters based on Tensor Train (TT) format. In this paper, we implement the TT-format representation for several RNN architectures such as simple RNN and Gated Recurrent Unit (GRU). We compare and evaluate our proposed RNN model with uncompressed RNN model on sequence classification and sequence prediction tasks. Our proposed RNNs with TT-format are able to preserve the performance while reducing the number of RNN parameters significantly up to 40 times smaller.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 3; 2024: 3; 2025: 3; 2026: 3.

28 papers.

Sort:

  1. 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 42
  2. 2021Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operatorscited by 28
  3. 2025CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activationcited by 28
  4. 2021QTN-VQC: An End-to-End Learning framework for Quantum Neural Networkscited by 27
  5. 2023Tensor networks for interpretable and efficient quantum-inspired machine learningcited by 16
  6. 2021Large-Scale Learning with Fourier Features and Tensor Decompositionscited by 14
  7. 2024CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimizationcited by 14
  8. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  9. 2024Guaranteed Nonconvex Factorization Approach for Tensor Train Recoverycited by 8
  10. 2024Federated Dynamical Low-Rank Training with Global Loss Convergence Guaranteescited by 7
  11. 2021Tensor-Train Recurrent Neural Networks for Interpretable Multi-Way Financial Forecastingcited by 6
  12. 2021Doping: A technique for efficient compression of LSTM models using sparse structured additive matricescited by 5
16 more
  1. 2021On the Memory Mechanism of Tensor-Power Recurrent Modelscited by 5
  2. 2025A Scalable Factorization Approach for High-Order Structured Tensor Recoverycited by 4
  3. 20213U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Accelerationcited by 3
  4. 2021Spectral Pruning for Recurrent Neural Networkscited by 3
  5. 2022Position: Tensor Networks are a Valuable Asset for Green AIcited by 3
  6. 2022Canonical convolutional neural networkscited by 3
  7. 2023Tetra-AML: Automatic Machine Learning via Tensor Networkscited by 3
  8. 2021Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directionscited by 2
  9. 2022TTML: tensor trains for general supervised machine learningcited by 2
  10. 2023Quantization-Aware and Tensor-Compressed Training of Transformers for Natural Language Understandingcited by 2
  11. 2026Compressing Transformer Language Models via Matrix Product Operator Decomposition: A Case Study on PicoGPTcited by 2
  12. 2022Error Analysis of Tensor-Train Cross Approximationcited by 1
  13. 2025Rec-AD: An Efficient Computation Framework for FDIA Detection Based on Tensor Train Decomposition and Deep Learning Recommendation Modelcited by 1
  14. 2022Embedding Compression for Text Classification Using Dictionary Screeningcited by 0
  15. 2026Quantum-inspired tensor networks in machine learning modelscited by 0
  16. 2026Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devicescited by 0