mlsanityData to 14 Oct 2026

Factorization tricks for LSTM networks

arXiv 1703.10722 ↗First posted 31 Mar 2017cs.CL cs.NE stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorization by design" of LSTM matrix into the product of two smaller matrices, and the second one is partitioning of LSTM matrix, its inputs and states into the independent groups. Both approaches allow us to train large LSTM networks significantly faster to the near state-of the art perplexity while using significantly less RNN parameters.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 1; 2023: 1; 2024: 4.

13 papers.

Sort:

By area:

  1. 2021Federated Learning Meets Natural Language Processing: A Surveycited by 43
  2. 2021Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNNcited by 16
  3. 2024Hierarchical State Space Models for Continuous Sequence-to-Sequence Modelingcited by 15
  4. 2022A new hope for network model generalizationcited by 11
  5. 2021Greenformer: Factorization Toolkit for Efficient Deep Neural Networkscited by 7
  6. 2024Generative AI Agents in Autonomous Machines: A Safety Perspectivecited by 7
  7. 2021Doping: A technique for efficient compression of LSTM models using sparse structured additive matricescited by 5
  8. 2021Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximationcited by 5
  9. 2021Attention-based Contextual Language Model Adaptation for Speech Recognitioncited by 4
  10. 2024A Survey of Distributed Learning in Cloud, Mobile, and Edge Settingscited by 2
  11. 2024IMSSA: Deploying modern state-space models on memristive in-memory compute hardwarecited by 2
  12. 2021Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classificationcited by 0
1 more
  1. 2023Extending Memory for Language Modellingcited by 0