{"list":"other","total":81,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2204.06125","title":"Hierarchical Text-Conditional Image Generation with CLIP Latents","created":"2022-04-13","cited_by_count":6585},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"1506.02142","title":"Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning","created":"2015-06-06","cited_by_count":4448},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"1510.00149","title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","created":"2015-10-01","cited_by_count":2329},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"1506.02626","title":"Learning both Weights and Connections for Efficient Neural Networks","created":"2015-06-08","cited_by_count":2031},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"1906.02243","title":"Energy and Policy Considerations for Deep Learning in NLP","created":"2019-06-05","cited_by_count":1254},{"id":"1608.08710","title":"Pruning Filters for Efficient ConvNets","created":"2016-08-31","cited_by_count":1143},{"id":"2301.00774","title":"SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot","created":"2023-01-02","cited_by_count":1121},{"id":"1905.09418","title":"Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned","created":"2019-05-23","cited_by_count":856},{"id":"1810.02340","title":"SNIP: Single-shot Network Pruning based on Connection Sensitivity","created":"2018-10-04","cited_by_count":812},{"id":"2106.05974","title":"Scaling Vision with Sparse Mixture of Experts","created":"2021-06-10","cited_by_count":747},{"id":"1912.05671","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","created":"2019-12-11","cited_by_count":637},{"id":"2003.03033","title":"What is the State of Neural Network Pruning?","created":"2020-03-06","cited_by_count":583},{"id":"1712.00409","title":"Deep Learning Scaling is Predictable, Empirically","created":"2017-12-01","cited_by_count":582},{"id":"2104.10350","title":"Carbon Emissions and Large Neural Network Training","created":"2021-04-21","cited_by_count":567},{"id":"1608.03665","title":"Learning Structured Sparsity in Deep Neural Networks","created":"2016-08-12","cited_by_count":541},{"id":"2102.00554","title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","created":"2021-01-31","cited_by_count":527},{"id":"1911.11134","title":"Rigging the Lottery: Making All Tickets Winners","created":"2019-11-25","cited_by_count":495},{"id":"2006.05467","title":"Pruning neural networks without any data by iteratively conserving synaptic flow","created":"2020-06-09","cited_by_count":479},{"id":"1902.09574","title":"The State of Sparsity in Deep Neural Networks","created":"2019-02-25","cited_by_count":476},{"id":"2002.07376","title":"Picking Winning Tickets Before Training by Preserving Gradient Flow","created":"2020-02-18","cited_by_count":471},{"id":"2012.07463","title":"Parameter-Efficient Transfer Learning with Diff Pruning","created":"2020-12-14","cited_by_count":352},{"id":"1707.04780","title":"Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science","created":"2017-07-15","cited_by_count":328},{"id":"2103.16716","title":"BASE Layers: Simplifying Training of Large, Sparse Models","created":"2021-03-30","cited_by_count":320},{"id":"2007.12223","title":"The Lottery Ticket Hypothesis for Pre-trained BERT Networks","created":"2020-07-23","cited_by_count":275},{"id":"2003.02389","title":"Comparing Rewinding and Fine-tuning in Neural Network Pruning","created":"2020-03-05","cited_by_count":240},{"id":"2106.04426","title":"Hash Layers For Large Sparse Models","created":"2021-06-08","cited_by_count":237},{"id":"1907.04840","title":"Sparse Networks from Scratch: Faster Training without Losing Performance","created":"2019-07-10","cited_by_count":213},{"id":"1902.05967","title":"Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization","created":"2019-02-15","cited_by_count":189},{"id":"2102.04010","title":"Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch","created":"2021-02-08","cited_by_count":179},{"id":"2111.09805","title":"DICE: Leveraging Sparsification for Out-of-Distribution Detection","created":"2021-11-18","cited_by_count":166},{"id":"1909.11957","title":"Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks","created":"2019-09-26","cited_by_count":160},{"id":"2006.10901","title":"Sparse GPU Kernels for Deep Learning","created":"2020-06-18","cited_by_count":141},{"id":"1711.05136","title":"Deep Rewiring: Training very sparse deep networks","created":"2017-11-14","cited_by_count":139},{"id":"2203.07259","title":"The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models","created":"2022-03-14","cited_by_count":115},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110},{"id":"2204.00595","title":"Monarch: Expressive Structured Matrices for Efficient and Accurate Training","created":"2022-04-01","cited_by_count":106},{"id":"1903.12561","title":"Adversarial Robustness vs Model Compression, or Both?","created":"2019-03-29","cited_by_count":98},{"id":"2210.14793","title":"M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design","created":"2022-10-26","cited_by_count":98},{"id":"2012.06908","title":"The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models","created":"2020-12-12","cited_by_count":91},{"id":"1911.09723","title":"Fast Sparse ConvNets","created":"2019-11-21","cited_by_count":86},{"id":"2202.02643","title":"The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training","created":"2022-02-05","cited_by_count":85},{"id":"2106.10404","title":"Sparse Training via Boosting Pruning Plasticity with Neuroregeneration","created":"2021-06-19","cited_by_count":84},{"id":"2203.12533","title":"Pathways: Asynchronous Distributed Dataflow for ML","created":"2022-03-23","cited_by_count":82},{"id":"2101.00063","title":"EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets","created":"2020-12-31","cited_by_count":79},{"id":"2106.03517","title":"Top-KAST: Top-K Always Sparse Training","created":"2021-06-07","cited_by_count":79},{"id":"2110.14032","title":"MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge","created":"2021-10-26","cited_by_count":79},{"id":"2010.03533","title":"Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win","created":"2020-10-07","cited_by_count":77},{"id":"1906.10732","title":"The Difficulty of Training Sparse Neural Networks","created":"2019-06-25","cited_by_count":62},{"id":"2106.09129","title":"A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness","created":"2021-06-16","cited_by_count":62},{"id":"2206.10369","title":"The State of Sparse Training in Deep Reinforcement Learning","created":"2022-06-17","cited_by_count":61},{"id":"2106.12379","title":"AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks","created":"2021-06-23","cited_by_count":59},{"id":"2210.04092","title":"Advancing Model Pruning via Bi-level Optimization","created":"2022-10-08","cited_by_count":56},{"id":"1708.08917","title":"CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices","created":"2017-08-29","cited_by_count":55},{"id":"2107.00166","title":"Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?","created":"2021-07-01","cited_by_count":52},{"id":"2111.13445","title":"How Well Do Sparse Imagenet Models Transfer?","created":"2021-11-26","cited_by_count":50},{"id":"2207.04630","title":"On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence","created":"2022-07-11","cited_by_count":50},{"id":"2106.04217","title":"Dynamic Sparse Training for Deep Reinforcement Learning","created":"2021-06-08","cited_by_count":49},{"id":"2110.00296","title":"Powerpropagation: A sparsity inducing weight reparameterisation","created":"2021-10-01","cited_by_count":49},{"id":"1901.09181","title":"Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware","created":"2019-01-26","cited_by_count":48},{"id":"2106.14568","title":"Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity","created":"2021-06-28","cited_by_count":44},{"id":"1901.09290","title":"PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration","created":"2019-01-26","cited_by_count":41},{"id":"2202.09844","title":"Sparsity Winning Twice: Better Robust Generalization from More Efficient Training","created":"2022-02-20","cited_by_count":41},{"id":"1604.05978","title":"A topological insight into restricted Boltzmann machines","created":"2016-04-20","cited_by_count":32},{"id":"2012.00560","title":"Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders","created":"2020-12-01","cited_by_count":28},{"id":"2202.04736","title":"Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets","created":"2022-02-09","cited_by_count":27},{"id":"2103.00397","title":"Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective","created":"2021-02-28","cited_by_count":25},{"id":"1901.07988","title":"Backprop with Approximate Activations for Memory-efficient Network Training","created":"2019-01-23","cited_by_count":19},{"id":"2001.07710","title":"An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices","created":"2020-01-20","cited_by_count":18},{"id":"2102.01732","title":"Truly Sparse Neural Networks at Scale","created":"2021-02-02","cited_by_count":16},{"id":"2301.05219","title":"Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning","created":"2023-01-12","cited_by_count":16},{"id":"2208.10842","title":"Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost","created":"2022-08-23","cited_by_count":11},{"id":"2209.15261","title":"Minimalistic Unsupervised Learning with the Sparse Manifold Transform","created":"2022-09-30","cited_by_count":6},{"id":"2206.04762","title":"Data-Efficient Double-Win Lottery Tickets from Robust Pre-training","created":"2022-06-09","cited_by_count":5},{"id":"2207.08006","title":"S4: a High-sparsity, High-performance AI Accelerator","created":"2022-07-16","cited_by_count":3}],"next_cursor":null}