{"list":"other","total":51,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"1906.02243","title":"Energy and Policy Considerations for Deep Learning in NLP","created":"2019-06-05","cited_by_count":1254},{"id":"1810.02340","title":"SNIP: Single-shot Network Pruning based on Connection Sensitivity","created":"2018-10-04","cited_by_count":812},{"id":"1708.06519","title":"Learning Efficient Convolutional Networks through Network Slimming","created":"2017-08-22","cited_by_count":676},{"id":"2012.13255","title":"Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning","created":"2020-12-22","cited_by_count":671},{"id":"1912.05671","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","created":"2019-12-11","cited_by_count":637},{"id":"1712.01312","title":"Learning Sparse Neural Networks through $L_0$ Regularization","created":"2017-12-04","cited_by_count":583},{"id":"2104.10350","title":"Carbon Emissions and Large Neural Network Training","created":"2021-04-21","cited_by_count":567},{"id":"2102.00554","title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","created":"2021-01-31","cited_by_count":527},{"id":"1911.11134","title":"Rigging the Lottery: Making All Tickets Winners","created":"2019-11-25","cited_by_count":495},{"id":"2006.05467","title":"Pruning neural networks without any data by iteratively conserving synaptic flow","created":"2020-06-09","cited_by_count":479},{"id":"1902.09574","title":"The State of Sparsity in Deep Neural Networks","created":"2019-02-25","cited_by_count":476},{"id":"2002.07376","title":"Picking Winning Tickets Before Training by Preserving Gradient Flow","created":"2020-02-18","cited_by_count":471},{"id":"1909.11556","title":"Reducing Transformer Depth on Demand with Structured Dropout","created":"2019-09-25","cited_by_count":439},{"id":"2005.07683","title":"Movement Pruning: Adaptive Sparsity by Fine-Tuning","created":"2020-05-15","cited_by_count":403},{"id":"1804.08838","title":"Measuring the Intrinsic Dimension of Objective Landscapes","created":"2018-04-24","cited_by_count":370},{"id":"1707.04780","title":"Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science","created":"2017-07-15","cited_by_count":328},{"id":"2103.16716","title":"BASE Layers: Simplifying Training of Large, Sparse Models","created":"2021-03-30","cited_by_count":320},{"id":"2007.12223","title":"The Lottery Ticket Hypothesis for Pre-trained BERT Networks","created":"2020-07-23","cited_by_count":275},{"id":"1907.04840","title":"Sparse Networks from Scratch: Faster Training without Losing Performance","created":"2019-07-10","cited_by_count":213},{"id":"1910.04732","title":"Structured Pruning of Large Language Models","created":"2019-10-10","cited_by_count":203},{"id":"1902.05967","title":"Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization","created":"2019-02-15","cited_by_count":189},{"id":"2009.08576","title":"Pruning Neural Networks at Initialization: Why are We Missing the Mark?","created":"2020-09-18","cited_by_count":182},{"id":"2102.04010","title":"Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch","created":"2021-02-08","cited_by_count":179},{"id":"1711.05136","title":"Deep Rewiring: Training very sparse deep networks","created":"2017-11-14","cited_by_count":139},{"id":"1810.05331","title":"Dynamic Channel Pruning: Feature Boosting and Suppression","created":"2018-10-12","cited_by_count":129},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110},{"id":"1906.06307","title":"A Signal Propagation Perspective for Pruning Neural Networks at Initialization","created":"2019-06-14","cited_by_count":96},{"id":"2101.00063","title":"EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets","created":"2020-12-31","cited_by_count":79},{"id":"2106.03517","title":"Top-KAST: Top-K Always Sparse Training","created":"2021-06-07","cited_by_count":79},{"id":"2010.03533","title":"Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win","created":"2020-10-07","cited_by_count":77},{"id":"2009.11094","title":"Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot","created":"2020-09-22","cited_by_count":66},{"id":"2106.12379","title":"AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks","created":"2021-06-23","cited_by_count":59},{"id":"1908.09979","title":"DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures","created":"2019-08-27","cited_by_count":56},{"id":"1906.03728","title":"The Generalization-Stability Tradeoff In Neural Network Pruning","created":"2019-06-09","cited_by_count":54},{"id":"1901.09181","title":"Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware","created":"2019-01-26","cited_by_count":48},{"id":"2004.14129","title":"How fine can fine-tuning be? Learning efficient language models","created":"2020-04-24","cited_by_count":41},{"id":"1711.02782","title":"Block-Sparse Recurrent Neural Networks","created":"2017-11-08","cited_by_count":39},{"id":"2009.11839","title":"A Gradient Flow Framework For Analyzing Network Pruning","created":"2020-09-24","cited_by_count":36},{"id":"2101.09048","title":"Selfish Sparse RNN Training","created":"2021-01-22","cited_by_count":26},{"id":"2102.01670","title":"Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization","created":"2021-02-02","cited_by_count":22},{"id":"1709.05027","title":"Learning Intrinsic Sparse Structures within Long Short-Term Memory","created":"2017-09-15","cited_by_count":18},{"id":"1912.00120","title":"One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation","created":"2019-11-30","cited_by_count":14},{"id":"1908.03463","title":"Group Pruning using a Bounded-Lp norm for Group Gating and Regularization","created":"2019-08-09","cited_by_count":3},{"id":"2006.13486","title":"Ramanujan Bipartite Graph Products for Efficient Block Sparse Neural Networks","created":"2020-06-24","cited_by_count":2},{"id":"2001.04686","title":"Block-wise Dynamic Sparseness","created":"2020-01-14","cited_by_count":1}],"next_cursor":null}