Transferability of Winning Lottery Tickets in Neural Network Differential Equation Solvers
Not on the map. Why?
Abstract (arXiv)
Recent work has shown that renormalisation group theory is a useful framework with which to describe the process of pruning neural networks via iterative magnitude pruning. This report formally describes the link between RG theory and IMP and extends previous results around the Lottery Ticket Hypothesis and Elastic Lottery Hypothesis to Hamiltonian Neural Networks for solving differential equations. We find lottery tickets for two Hamiltonian Neural Networks and demonstrate transferability between the two systems, with accuracy being dependent on integration times. The universality of the two systems is then analysed using tools from an RG perspective.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 16 Jun 2023.
Builds on
35 of 56 references.
References · 35
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
- 2019A Comprehensive Survey on Transfer Learningcited by 1,160
- 2019Are Sixteen Heads Really Better than One?cited by 816
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
- 2014Deep Learning in Neural Networks: An Overviewcited by 770
- 2019Hamiltonian Neural Networkscited by 712
- 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
- 2017Solving high-dimensional partial differential equations using deep learningcited by 556
- 2019The State of Sparsity in Deep Neural Networkscited by 476
- 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
- 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
23 more
- 2017A unified deep artificial neural network approach to partial differential equations in complex geometriescited by 122
- 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
- 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
- 2020Methods for Pruning Deep Neural Networkscited by 49
- 2021Hessian-Aware Pruning and Optimal Neural Implantcited by 45
- 2020Hamiltonian neural networks for solving equations of motioncited by 33
- 2020Optimizing Neural Networks via Koopman Operator Theorycited by 33
- 2020On the Predictability of Pruning Across Scalescited by 33
- 2020Robust Pruning at Initializationcited by 31
- 2021On the Existence of Universal Lottery Ticketscited by 30
- 2019Sparse Transfer Learning via Winning Lottery Ticketscited by 27
- 2020Pruning Algorithms to Accelerate Convolutional Neural Networks for Edge Applications: A Surveycited by 26
- 2020Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruningcited by 23
- 2021The Elastic Lottery Ticket Hypothesiscited by 23
- 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
- 2021Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Wincited by 11
- 2013Big Neural Networks Waste Capacitycited by 10
- 2020Space-time deep neural network approximations for high-dimensional partial differential equationscited by 9
- 2021Universality of Winning Tickets: A Renormalization Group Perspectivecited by 8
- 2020Dynamical Systems and Neural Networkscited by 5
- 2021dNNsolve: an efficient NN-based PDE solvercited by 4
- 2022Playing Lottery Tickets in Style Transfer Modelscited by 1
- 2023Considering Layerwise Importance in the Lottery Ticket Hypothesiscited by 1
Built on by
Not cited yet.