mlsanityData to 14 Oct 2026

Graph Expansion in Pruned Recurrent Neural Network Layers Preserve Performance

arXiv 2403.11100 ↗First posted 17 Mar 2024cs.LG cs.CV cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Expansion property of a graph refers to its strong connectivity as well as sparseness. It has been reported that deep neural networks can be pruned to a high degree of sparsity while maintaining their performance. Such pruning is essential for performing real time sequence learning tasks using recurrent neural networks in resource constrained platforms. We prune recurrent networks such as RNNs and LSTMs, maintaining a large spectral gap of the underlying graphs and ensuring their layerwise expansion properties. We also study the time unfolded recurrent network graphs in terms of the properties of their bipartite layers. Experimental results for the benchmark sequence MNIST, CIFAR-10, and Google speech command data show that expander graph properties are key to preserving classification accuracy of RNN and LSTM.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 54 references that are in the map. First posted on arXiv 17 Mar 2024.

Builds on

26 of 54 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  9. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  10. 2020What is the State of Neural Network Pruning?cited by 583
  11. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  12. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
14 more
  1. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  2. 2017Dilated Recurrent Neural Networkscited by 83
  3. 2017Exploring Sparsity in Recurrent Neural Networkscited by 75
  4. 2020Graph Structure of Neural Networkscited by 68
  5. 2016ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGAcited by 52
  6. 2021From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformerscited by 38
  7. 2017Deep Expander Networks: Efficient Deep Networks from Graph Theorycited by 23
  8. 2019One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluationcited by 14
  9. 2018Efficient Recurrent Neural Networks using Structured Matrices in FPGAscited by 4
  10. 2016Sparse Activity and Sparse Connectivity in Supervised Learningcited by 3
  11. 2019Intrinsically Sparse Long Short-Term Memory Networkscited by 3
  12. 2021Spectral Pruning for Recurrent Neural Networkscited by 3
  13. 2017Sparse Neural Networks Topologiescited by 2
  14. 2022Sparse Super-Regular Networkscited by 1

Built on by

Not cited yet.