{"list":"other","total":57,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"1503.02531","title":"Distilling the Knowledge in a Neural Network","created":"2015-03-09","cited_by_count":10324},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2309.16609","title":"Qwen Technical Report","created":"2023-09-28","cited_by_count":3344},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1909.11942","title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","created":"2019-09-26","cited_by_count":2790},{"id":"1510.00149","title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","created":"2015-10-01","cited_by_count":2329},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"1506.02626","title":"Learning both Weights and Connections for Efficient Neural Networks","created":"2015-06-08","cited_by_count":2031},{"id":"2210.17323","title":"GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers","created":"2022-10-31","cited_by_count":1919},{"id":"2404.14219","title":"Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone","created":"2024-04-22","cited_by_count":1917},{"id":"2406.01574","title":"MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark","created":"2024-06-03","cited_by_count":1848},{"id":"2401.04088","title":"Mixtral of Experts","created":"2024-01-08","cited_by_count":1823},{"id":"2211.17192","title":"Fast Inference from Transformers via Speculative Decoding","created":"2022-11-30","cited_by_count":1653},{"id":"2301.00774","title":"SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot","created":"2023-01-02","cited_by_count":1121},{"id":"2305.14233","title":"Enhancing Chat Language Models by Scaling High-quality Instructional Conversations","created":"2023-05-23","cited_by_count":839},{"id":"2306.14048","title":"H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models","created":"2023-06-24","cited_by_count":832},{"id":"2306.11695","title":"A Simple and Effective Pruning Approach for Large Language Models","created":"2023-06-20","cited_by_count":800},{"id":"2305.11627","title":"LLM-Pruner: On the Structural Pruning of Large Language Models","created":"2023-05-19","cited_by_count":650},{"id":"1807.03819","title":"Universal Transformers","created":"2018-07-10","cited_by_count":647},{"id":"1912.05671","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","created":"2019-12-11","cited_by_count":637},{"id":"2404.00456","title":"QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs","created":"2024-03-30","cited_by_count":634},{"id":"2102.00554","title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","created":"2021-01-31","cited_by_count":527},{"id":"1902.09574","title":"The State of Sparsity in Deep Neural Networks","created":"2019-02-25","cited_by_count":476},{"id":"2106.04647","title":"Compacter: Efficient Low-Rank Hypercomplex Adapter Layers","created":"2021-06-08","cited_by_count":460},{"id":"2305.17118","title":"Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time","created":"2023-05-26","cited_by_count":439},{"id":"2406.02069","title":"PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling","created":"2024-06-04","cited_by_count":432},{"id":"2005.07683","title":"Movement Pruning: Adaptive Sparsity by Fine-Tuning","created":"2020-05-15","cited_by_count":403},{"id":"2401.15024","title":"SliceGPT: Compress Large Language Models by Deleting Rows and Columns","created":"2024-01-26","cited_by_count":385},{"id":"2403.03853","title":"ShortGPT: Layers in Large Language Models are More Redundant Than You Expect","created":"2024-03-06","cited_by_count":383},{"id":"2406.16858","title":"EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees","created":"2024-06-24","cited_by_count":370},{"id":"2403.12968","title":"LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression","created":"2024-03-19","cited_by_count":348},{"id":"2208.11580","title":"Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning","created":"2022-08-24","cited_by_count":322},{"id":"1404.0736","title":"Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation","created":"2014-04-02","cited_by_count":306},{"id":"1312.4314","title":"Learning Factored Representations in a Deep Mixture of Experts","created":"2013-12-16","cited_by_count":278},{"id":"2104.08378","title":"Accelerating Sparse Deep Neural Networks","created":"2021-04-16","cited_by_count":223},{"id":"2402.14905","title":"MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases","created":"2024-02-22","cited_by_count":223},{"id":"2209.01667","title":"A Review of Sparse Expert Models in Deep Learning","created":"2022-09-04","cited_by_count":148},{"id":"2407.14679","title":"Compact Language Models via Pruning and Knowledge Distillation","created":"2024-07-19","cited_by_count":148},{"id":"2108.06098","title":"FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning","created":"2021-08-13","cited_by_count":141},{"id":"2402.11187","title":"LaCo: Large Language Model Pruning via Layer Collapse","created":"2024-02-17","cited_by_count":126},{"id":"2002.02925","title":"BERT-of-Theseus: Compressing BERT by Progressive Module Replacing","created":"2020-02-07","cited_by_count":113},{"id":"2309.14859","title":"Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation","created":"2023-09-26","cited_by_count":109},{"id":"2406.15786","title":"What Matters in Transformers? Not All Attention is Needed","created":"2024-06-22","cited_by_count":84},{"id":"2402.11295","title":"OneBit: Towards Extremely Low-bit Large Language Models","created":"2024-02-17","cited_by_count":79},{"id":"2410.20672","title":"Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA","created":"2024-10-28","cited_by_count":78},{"id":"1906.02768","title":"Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP","created":"2019-06-06","cited_by_count":73},{"id":"1906.05392","title":"Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian","created":"2019-06-12","cited_by_count":67},{"id":"2407.16286","title":"A deeper look at depth pruning of LLMs","created":"2024-07-23","cited_by_count":23},{"id":"1602.02262","title":"Recovery guarantee of weighted low-rank approximation via alternating minimization","created":"2016-02-06","cited_by_count":20},{"id":"2410.07170","title":"Parameter Efficient Fine-tuning via Explained Variance Adaptation","created":"2024-10-09","cited_by_count":20},{"id":"2009.05647","title":"Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximation","created":"2020-09-11","cited_by_count":8},{"id":"2310.02489","title":"ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers","created":"2023-10-03","cited_by_count":4}],"next_cursor":null}