{"list":"other","total":157,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2304.08485","title":"Visual Instruction Tuning","created":"2023-04-17","cited_by_count":7869},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2309.06180","title":"Efficient Memory Management for Large Language Model Serving with PagedAttention","created":"2023-09-12","cited_by_count":5944},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"2312.00752","title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces","created":"2023-12-01","cited_by_count":4718},{"id":"2101.00190","title":"Prefix-Tuning: Optimizing Continuous Prompts for Generation","created":"2021-01-01","cited_by_count":3915},{"id":"2104.08691","title":"The Power of Scale for Parameter-Efficient Prompt Tuning","created":"2021-04-18","cited_by_count":3795},{"id":"2205.14135","title":"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","created":"2022-05-27","cited_by_count":3673},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2305.14314","title":"QLoRA: Efficient Finetuning of Quantized LLMs","created":"2023-05-23","cited_by_count":3418},{"id":"2004.05150","title":"Longformer: The Long-Document Transformer","created":"2020-04-10","cited_by_count":3413},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"2307.08691","title":"FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning","created":"2023-07-17","cited_by_count":2598},{"id":"2303.18223","title":"A Survey of Large Language Models","created":"2023-03-31","cited_by_count":2576},{"id":"2111.00396","title":"Efficiently Modeling Long Sequences with Structured State Spaces","created":"2021-10-31","cited_by_count":2465},{"id":"1510.00149","title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","created":"2015-10-01","cited_by_count":2329},{"id":"1910.03771","title":"HuggingFace's Transformers: State-of-the-art Natural Language Processing","created":"2019-10-09","cited_by_count":2218},{"id":"1909.08053","title":"Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism","created":"2019-09-17","cited_by_count":1868},{"id":"2401.04088","title":"Mixtral of Experts","created":"2024-01-08","cited_by_count":1823},{"id":"2007.14062","title":"Big Bird: Transformers for Longer Sequences","created":"2020-07-28","cited_by_count":1684},{"id":"2211.17192","title":"Fast Inference from Transformers via Speculative Decoding","created":"2022-11-30","cited_by_count":1653},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"1712.05877","title":"Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference","created":"2017-12-15","cited_by_count":1593},{"id":"1609.08144","title":"Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation","created":"2016-09-26","cited_by_count":1475},{"id":"2006.04768","title":"Linformer: Self-Attention with Linear Complexity","created":"2020-06-08","cited_by_count":1474},{"id":"2211.10438","title":"SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models","created":"2022-11-18","cited_by_count":1386},{"id":"2305.13245","title":"GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints","created":"2023-05-22","cited_by_count":1346},{"id":"2405.21060","title":"Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality","created":"2024-05-31","cited_by_count":1336},{"id":"1910.02054","title":"ZeRO: Memory Optimizations Toward Training Trillion Parameter Models","created":"2019-10-04","cited_by_count":1301},{"id":"2306.00978","title":"AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration","created":"2023-06-01","cited_by_count":1298},{"id":"2012.15723","title":"Making Pre-trained Language Models Better Few-shot Learners","created":"2020-12-31","cited_by_count":1256},{"id":"1710.03740","title":"Mixed Precision Training","created":"2017-10-10","cited_by_count":1156},{"id":"2301.00774","title":"SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot","created":"2023-01-02","cited_by_count":1121},{"id":"2103.03206","title":"Perceiver: General Perception with Iterative Attention","created":"2021-03-04","cited_by_count":1104},{"id":"2001.07676","title":"Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference","created":"2020-01-21","cited_by_count":938},{"id":"1603.09320","title":"Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs","created":"2016-03-30","cited_by_count":861},{"id":"1604.06174","title":"Training Deep Nets with Sublinear Memory Cost","created":"2016-04-21","cited_by_count":848},{"id":"2208.04933","title":"Simplified State Space Layers for Sequence Modeling","created":"2022-08-09","cited_by_count":788},{"id":"2106.05974","title":"Scaling Vision with Sparse Mixture of Experts","created":"2021-06-10","cited_by_count":747},{"id":"1911.02150","title":"Fast Transformer Decoding: One Write-Head is All You Need","created":"2019-11-06","cited_by_count":717},{"id":"2103.10385","title":"GPT Understands, Too","created":"2021-03-18","cited_by_count":716},{"id":"2110.07602","title":"P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks","created":"2021-10-14","cited_by_count":689},{"id":"2305.11627","title":"LLM-Pruner: On the Structural Pruning of Large Language Models","created":"2023-05-19","cited_by_count":650},{"id":"2404.00456","title":"QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs","created":"2024-03-30","cited_by_count":634},{"id":"2110.13985","title":"Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers","created":"2021-10-26","cited_by_count":608},{"id":"2008.07669","title":"HiPPO: Recurrent Memory with Optimal Polynomial Projections","created":"2020-08-17","cited_by_count":605},{"id":"2303.06865","title":"FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU","created":"2023-03-13","cited_by_count":575},{"id":"2311.18677","title":"Splitwise: Efficient generative LLM inference using phase splitting","created":"2023-11-30","cited_by_count":558},{"id":"2403.02310","title":"Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve","created":"2024-03-04","cited_by_count":526},{"id":"2307.13304","title":"QuIP: 2-Bit Quantization of Large Language Models With Guarantees","created":"2023-07-25","cited_by_count":414},{"id":"2207.00032","title":"DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale","created":"2022-06-30","cited_by_count":407},{"id":"2403.15388","title":"LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models","created":"2024-03-22","cited_by_count":402},{"id":"2403.19887","title":"Jamba: A Hybrid Transformer-Mamba Language Model","created":"2024-03-28","cited_by_count":401},{"id":"2202.08906","title":"ST-MoE: Designing Stable and Transferable Sparse Expert Models","created":"2022-02-17","cited_by_count":399},{"id":"2104.06599","title":"Learning How to Ask: Querying LMs with Mixtures of Soft Prompts","created":"2021-04-14","cited_by_count":377},{"id":"1502.02551","title":"Deep Learning with Limited Numerical Precision","created":"2015-02-09","cited_by_count":376},{"id":"2305.09781","title":"SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification","created":"2023-05-16","cited_by_count":365},{"id":"2103.02143","title":"Random Feature Attention","created":"2021-03-03","cited_by_count":354},{"id":"2012.07463","title":"Parameter-Efficient Transfer Learning with Diff Pruning","created":"2020-12-14","cited_by_count":352},{"id":"2101.06840","title":"ZeRO-Offload: Democratizing Billion-Scale Model Training","created":"2021-01-18","cited_by_count":341},{"id":"2405.02246","title":"What matters when building vision-language models?","created":"2024-05-03","cited_by_count":326},{"id":"2103.16716","title":"BASE Layers: Simplifying Training of Large, Sparse Models","created":"2021-03-30","cited_by_count":320},{"id":"2104.05240","title":"Factual Probing Is [MASK]: Learning vs. Learning to Recall","created":"2021-04-12","cited_by_count":291},{"id":"1312.4314","title":"Learning Factored Representations in a Deep Mixture of Experts","created":"2013-12-16","cited_by_count":278},{"id":"2209.15425","title":"Spikformer: When Spiking Neural Network Meets Transformer","created":"2022-09-29","cited_by_count":271},{"id":"2404.02258","title":"Mixture-of-Depths: Dynamically allocating compute in transformer-based language models","created":"2024-04-02","cited_by_count":258},{"id":"2104.07857","title":"ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning","created":"2021-04-16","cited_by_count":256},{"id":"2202.01169","title":"Unified Scaling Laws for Routed Language Models","created":"2022-02-02","cited_by_count":253},{"id":"2310.17157","title":"Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time","created":"2023-10-26","cited_by_count":252},{"id":"2308.00951","title":"From Sparse to Soft Mixtures of Experts","created":"2023-08-02","cited_by_count":248},{"id":"2106.04426","title":"Hash Layers For Large Sparse Models","created":"2021-06-08","cited_by_count":237},{"id":"2012.09852","title":"SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning","created":"2020-12-17","cited_by_count":234},{"id":"2101.00121","title":"WARP: Word-level Adversarial ReProgramming","created":"2021-01-01","cited_by_count":221},{"id":"2312.12456","title":"PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU","created":"2023-12-16","cited_by_count":217},{"id":"2111.09839","title":"Training Neural Networks with Fixed Sparse Masks","created":"2021-11-18","cited_by_count":214},{"id":"2404.14294","title":"A Survey on Efficient Inference for Large Language Models","created":"2024-04-22","cited_by_count":205},{"id":"2304.01433","title":"TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings","created":"2023-04-04","cited_by_count":202},{"id":"2308.16369","title":"SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills","created":"2023-08-31","cited_by_count":193},{"id":"2312.11514","title":"LLM in a flash: Efficient Large Language Model Inference with Limited Memory","created":"2023-12-12","cited_by_count":177},{"id":"2310.08461","title":"DistillSpec: Improving Speculative Decoding via Knowledge Distillation","created":"2023-10-12","cited_by_count":165},{"id":"2302.07863","title":"Speculative Decoding with Big Little Decoder","created":"2023-02-15","cited_by_count":164},{"id":"2309.14717","title":"QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models","created":"2023-09-26","cited_by_count":155},{"id":"2110.01786","title":"MoEfication: Transformer Feed-forward Layers are Mixtures of Experts","created":"2021-10-05","cited_by_count":152},{"id":"2305.05920","title":"Fast Distributed Inference Serving for Large Language Models","created":"2023-05-10","cited_by_count":145},{"id":"2310.15141","title":"SpecTr: Fast Speculative Decoding via Optimal Transport","created":"2023-10-23","cited_by_count":129},{"id":"2406.16554","title":"LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training","created":"2024-06-24","cited_by_count":123},{"id":"2401.11181","title":"Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads","created":"2024-01-20","cited_by_count":121},{"id":"1806.03377","title":"PipeDream: Fast and Efficient Pipeline Parallel DNN Training","created":"2018-06-08","cited_by_count":117},{"id":"2310.07177","title":"Online Speculative Decoding","created":"2023-10-11","cited_by_count":113},{"id":"1712.01507","title":"Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks","created":"2017-12-05","cited_by_count":109}],"next_cursor":"100"}