Balancing Sparse RNNs with Hyperparameterization Benefiting Meta-Learning
Not on the map. Why?
Abstract (arXiv)
This paper develops alternative hyperparameters for specifying sparse Recurrent Neural Networks (RNNs). These hyperparameters allow for varying sparsity within the trainable weight matrices of the model while improving overall performance. This architecture enables the definition of a novel metric, hidden proportion, which seeks to balance the distribution of unknowns within the model and provides significant explanatory power of model performance. Together, the use of the varied sparsity RNN architecture combined with the hidden proportion metric generates significant performance gains while improving performance expectations on an a priori basis. This combined approach provides a path forward towards generalized meta-learning applications and model optimization based on intrinsic characteristics of the data set, including input and output dimensions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 18 Sep 2025.
Builds on
12 of 22 references.
References · 12
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
- 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
- 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
- 2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 182
- 2020Hyper-Parameter Optimization: A Review of Algorithms and Applicationscited by 114
- 2017Block-Sparse Recurrent Neural Networkscited by 39
- 2022A Comparative study of Hyper-Parameter Optimization Toolscited by 10
- 2019RNN Architecture Learning with Sparse Regularizationcited by 4
- 2024Rethinking the Relationship between Recurrent and Non-Recurrent Neural Networks: A Study in Sparsitycited by 3
- 2024Investigating Sparsity in Recurrent Neural Networkscited by 1
Built on by
Not cited yet.