{"list":"other","total":63,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"1510.00149","title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","created":"2015-10-01","cited_by_count":2329},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"2307.02483","title":"Jailbroken: How Does LLM Safety Training Fail?","created":"2023-07-05","cited_by_count":1651},{"id":"1904.10509","title":"Generating Long Sequences with Sparse Transformers","created":"2019-04-23","cited_by_count":1546},{"id":"1906.02243","title":"Energy and Policy Considerations for Deep Learning in NLP","created":"2019-06-05","cited_by_count":1254},{"id":"2301.00774","title":"SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot","created":"2023-01-02","cited_by_count":1121},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"1810.03993","title":"Model Cards for Model Reporting","created":"2018-10-05","cited_by_count":1080},{"id":"2004.09456","title":"StereoSet: Measuring stereotypical bias in pretrained language models","created":"2020-04-20","cited_by_count":924},{"id":"1804.04235","title":"Adafactor: Adaptive Learning Rates with Sublinear Memory Cost","created":"2018-04-11","cited_by_count":905},{"id":"1810.02340","title":"SNIP: Single-shot Network Pruning based on Connection Sensitivity","created":"2018-10-04","cited_by_count":812},{"id":"1710.01878","title":"To prune, or not to prune: exploring the efficacy of pruning for model compression","created":"2017-10-05","cited_by_count":603},{"id":"1810.05270","title":"Rethinking the Value of Network Pruning","created":"2018-10-11","cited_by_count":599},{"id":"2104.10350","title":"Carbon Emissions and Large Neural Network Training","created":"2021-04-21","cited_by_count":567},{"id":"1909.01326","title":"The Woman Worked as a Babysitter: On Biases in Language Generation","created":"2019-09-03","cited_by_count":541},{"id":"2102.00554","title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","created":"2021-01-31","cited_by_count":527},{"id":"1911.11134","title":"Rigging the Lottery: Making All Tickets Winners","created":"2019-11-25","cited_by_count":495},{"id":"2006.05467","title":"Pruning neural networks without any data by iteratively conserving synaptic flow","created":"2020-06-09","cited_by_count":479},{"id":"1902.09574","title":"The State of Sparsity in Deep Neural Networks","created":"2019-02-25","cited_by_count":476},{"id":"2002.07376","title":"Picking Winning Tickets Before Training by Preserving Gradient Flow","created":"2020-02-18","cited_by_count":471},{"id":"1707.04780","title":"Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science","created":"2017-07-15","cited_by_count":328},{"id":"1903.03862","title":"Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them","created":"2019-03-09","cited_by_count":268},{"id":"2310.17157","title":"Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time","created":"2023-10-26","cited_by_count":252},{"id":"1911.13299","title":"What's Hidden in a Randomly Weighted Neural Network?","created":"2019-11-29","cited_by_count":237},{"id":"2104.08378","title":"Accelerating Sparse Deep Neural Networks","created":"2021-04-16","cited_by_count":223},{"id":"1802.08435","title":"Efficient Neural Audio Synthesis","created":"2018-02-23","cited_by_count":219},{"id":"1907.04840","title":"Sparse Networks from Scratch: Faster Training without Losing Performance","created":"2019-07-10","cited_by_count":213},{"id":"2011.14522","title":"Feature Learning in Infinite-Width Neural Networks","created":"2020-11-30","cited_by_count":190},{"id":"2009.08576","title":"Pruning Neural Networks at Initialization: Why are We Missing the Mark?","created":"2020-09-18","cited_by_count":182},{"id":"1911.05248","title":"What Do Compressed Deep Neural Networks Forget?","created":"2019-11-13","cited_by_count":171},{"id":"2010.03058","title":"Characterising Bias in Compressed Models","created":"2020-10-06","cited_by_count":167},{"id":"1711.05136","title":"Deep Rewiring: Training very sparse deep networks","created":"2017-11-14","cited_by_count":139},{"id":"1711.04735","title":"Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice","created":"2017-11-13","cited_by_count":132},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110},{"id":"2310.17813","title":"A Spectral Condition for Feature Learning","created":"2023-10-26","cited_by_count":109},{"id":"1906.06307","title":"A Signal Propagation Perspective for Pruning Neural Networks at Initialization","created":"2019-06-14","cited_by_count":96},{"id":"2202.02643","title":"The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training","created":"2022-02-05","cited_by_count":85},{"id":"2010.03533","title":"Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win","created":"2020-10-07","cited_by_count":77},{"id":"1810.09619","title":"Sparse DNNs with Improved Adversarial Robustness","created":"2018-10-23","cited_by_count":72},{"id":"2309.16620","title":"Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit","created":"2023-09-28","cited_by_count":66},{"id":"2006.09081","title":"Progressive Skeletonization: Trimming more fat from a network at initialization","created":"2020-06-16","cited_by_count":64},{"id":"2309.08520","title":"Scaling Laws for Sparsely-Connected Foundation Models","created":"2023-09-15","cited_by_count":43},{"id":"2305.02299","title":"Dynamic Sparse Training with Structured Sparsity","created":"2023-05-03","cited_by_count":38},{"id":"2009.11839","title":"A Gradient Flow Framework For Analyzing Network Pruning","created":"2020-09-24","cited_by_count":36},{"id":"1604.05978","title":"A topological insight into restricted Boltzmann machines","created":"2016-04-20","cited_by_count":32},{"id":"2303.10464","title":"SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models","created":"2023-03-18","cited_by_count":28},{"id":"1811.00206","title":"Balanced Sparsity for Efficient DNN Inference on GPU","created":"2018-11-01","cited_by_count":27},{"id":"2010.14495","title":"Are wider nets better given the same number of parameters?","created":"2020-10-27","cited_by_count":27},{"id":"2006.00896","title":"Pruning via Iterative Ranking of Sensitivity Statistics","created":"2020-06-01","cited_by_count":26},{"id":"1804.09862","title":"Accelerator-Aware Pruning for Convolutional Neural Networks","created":"2018-04-26","cited_by_count":22},{"id":"2102.01670","title":"Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization","created":"2021-02-02","cited_by_count":22},{"id":"2309.11568","title":"BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model","created":"2023-09-20","cited_by_count":19},{"id":"2402.04744","title":"Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers","created":"2024-02-07","cited_by_count":18},{"id":"2308.02060","title":"Accurate Neural Network Pruning Requires Rethinking Sparse Optimization","created":"2023-08-03","cited_by_count":10},{"id":"2303.11525","title":"Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency","created":"2023-03-21","cited_by_count":8},{"id":"2306.03725","title":"Towards Memory-Efficient Training for Extremely Large Output Spaces -- Learning with 500k Labels on a Single Commodity GPU","created":"2023-06-06","cited_by_count":6}],"next_cursor":null}