Nerva: a Truly Sparse Implementation of Neural Networks
Not on the map. Why?
Abstract (arXiv)
We introduce Nerva, a fast neural network library under development in C++. It supports sparsity by using the sparse matrix operations of Intel's Math Kernel Library (MKL), which eliminates the need for binary masks. We show that Nerva significantly decreases training time and memory usage while reaching equivalent accuracy to PyTorch. We run static sparse experiments with an MLP on CIFAR-10. On high sparsity levels like 99%, the runtime is reduced by a factor of 4× compared to a PyTorch model using masks. Similar to other popular frameworks such as PyTorch and Keras, Nerva offers a Python interface for users to work with.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 24 Jul 2024.
Builds on
22 of 29 references.
References · 22
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
- 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
- 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
- 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 325
- 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
- 2019What's Hidden in a Randomly Weighted Neural Network?cited by 236
- 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 210
- 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
- 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
10 more
- 2020Sparse GPU Kernels for Deep Learningcited by 138
- 2017Deep Rewiring: Training very sparse deep networkscited by 136
- 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
- 2019Fast Sparse ConvNetscited by 86
- 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
- 2019Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardwarecited by 48
- 2022Rare Gems: Finding Lottery Tickets at Initializationcited by 34
- 2016A topological insight into restricted Boltzmann machinescited by 32
- 2021Truly Sparse Neural Networks at Scalecited by 15
- 2018DLL: A Blazing Fast Deep Neural Network Librarycited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.