{"list":"other","total":90,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"2305.14314","title":"QLoRA: Efficient Finetuning of Quantized LLMs","created":"2023-05-23","cited_by_count":3418},{"id":"2309.16609","title":"Qwen Technical Report","created":"2023-09-28","cited_by_count":3344},{"id":"2310.06825","title":"Mistral 7B","created":"2023-10-10","cited_by_count":3164},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"2408.00118","title":"Gemma 2: Improving Open Language Models at a Practical Size","created":"2024-07-31","cited_by_count":1998},{"id":"2408.03314","title":"Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters","created":"2024-08-06","cited_by_count":1986},{"id":"2211.05100","title":"BLOOM: A 176B-Parameter Open-Access Multilingual Language Model","created":"2022-11-09","cited_by_count":1984},{"id":"2210.17323","title":"GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers","created":"2022-10-31","cited_by_count":1919},{"id":"1909.08053","title":"Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism","created":"2019-09-17","cited_by_count":1868},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"1712.05877","title":"Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference","created":"2017-12-15","cited_by_count":1593},{"id":"2211.10438","title":"SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models","created":"2022-11-18","cited_by_count":1386},{"id":"2306.00978","title":"AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration","created":"2023-06-01","cited_by_count":1298},{"id":"2112.11446","title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","created":"2021-12-08","cited_by_count":1202},{"id":"2212.07143","title":"Reproducible scaling laws for contrastive language-image learning","created":"2022-12-14","cited_by_count":1173},{"id":"1710.03740","title":"Mixed Precision Training","created":"2017-10-10","cited_by_count":1156},{"id":"2407.21787","title":"Large Language Monkeys: Scaling Inference Compute with Repeated Sampling","created":"2024-07-31","cited_by_count":991},{"id":"2305.06161","title":"StarCoder: may the source be with you!","created":"2023-05-09","cited_by_count":799},{"id":"2402.00838","title":"OLMo: Accelerating the Science of Language Models","created":"2024-02-01","cited_by_count":613},{"id":"2303.06865","title":"FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU","created":"2023-03-13","cited_by_count":575},{"id":"2402.19173","title":"StarCoder 2 and The Stack v2: The Next Generation","created":"2024-02-29","cited_by_count":502},{"id":"2104.08758","title":"Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus","created":"2021-04-18","cited_by_count":492},{"id":"2402.00159","title":"Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research","created":"2024-01-31","cited_by_count":483},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"2307.13304","title":"QuIP: 2-Bit Quantization of Large Language Models With Guarantees","created":"2023-07-25","cited_by_count":414},{"id":"2110.02861","title":"8-bit Optimizers via Block-wise Quantization","created":"2021-10-06","cited_by_count":403},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403},{"id":"2406.11794","title":"DataComp-LM: In search of the next generation of training sets for language models","created":"2024-06-17","cited_by_count":398},{"id":"2306.03078","title":"SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression","created":"2023-06-05","cited_by_count":367},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363},{"id":"2305.17888","title":"LLM-QAT: Data-Free Quantization Aware Training for Large Language Models","created":"2023-05-29","cited_by_count":324},{"id":"2402.17764","title":"The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits","created":"2024-02-27","cited_by_count":308},{"id":"2409.02060","title":"OLMoE: Open Mixture-of-Experts Language Models","created":"2024-09-03","cited_by_count":301},{"id":"2402.07827","title":"Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model","created":"2024-02-12","cited_by_count":295},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"2308.07633","title":"A Survey on Model Compression for Large Language Models","created":"2023-08-15","cited_by_count":269},{"id":"2202.01169","title":"Unified Scaling Laws for Routed Language Models","created":"2022-02-02","cited_by_count":253},{"id":"2212.09720","title":"The case for 4-bit precision: k-bit Inference Scaling Laws","created":"2022-12-19","cited_by_count":251},{"id":"2402.16827","title":"A Survey on Data Selection for Language Models","created":"2024-02-26","cited_by_count":236},{"id":"2209.05433","title":"FP8 Formats for Deep Learning","created":"2022-09-12","cited_by_count":196},{"id":"2310.11453","title":"BitNet: Scaling 1-bit Transformers for Large Language Models","created":"2023-10-17","cited_by_count":189},{"id":"2004.09602","title":"Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation","created":"2020-04-20","cited_by_count":188},{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183},{"id":"2407.01492","title":"RegMix: Data Mixture as Regression for Language Model Pre-training","created":"2024-07-01","cited_by_count":177},{"id":"2402.07871","title":"Scaling Laws for Fine-Grained Mixture of Experts","created":"2024-02-12","cited_by_count":173},{"id":"2301.03988","title":"SantaCoder: don't reach for the stars!","created":"2023-01-09","cited_by_count":170},{"id":"2405.18392","title":"Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations","created":"2024-05-28","cited_by_count":170},{"id":"2401.00448","title":"Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws","created":"2023-12-31","cited_by_count":159},{"id":"2402.04291","title":"BiLLM: Pushing the Limit of Post-Training Quantization for LLMs","created":"2024-02-06","cited_by_count":144},{"id":"2404.05405","title":"Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws","created":"2024-04-08","cited_by_count":144},{"id":"2202.08658","title":"The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks","created":"2022-02-17","cited_by_count":142},{"id":"2301.03728","title":"Scaling Laws for Generative Mixed-Modal Language Models","created":"2023-01-10","cited_by_count":134},{"id":"2209.06640","title":"Revisiting Neural Scaling Laws in Language and Vision","created":"2022-09-13","cited_by_count":132},{"id":"2405.10938","title":"Observational Scaling Laws and the Predictability of Language Model Performance","created":"2024-05-17","cited_by_count":130},{"id":"2407.13623","title":"Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies","created":"2024-07-18","cited_by_count":118},{"id":"2403.08540","title":"Language models scale reliably with over-training and on downstream tasks","created":"2024-03-13","cited_by_count":113},{"id":"2402.01092","title":"A Dynamical Model of Neural Scaling Laws","created":"2024-02-02","cited_by_count":111},{"id":"2312.04567","title":"Scaling Laws of Synthetic Images for Model Training ... for Now","created":"2023-12-07","cited_by_count":108},{"id":"2207.10551","title":"Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?","created":"2022-07-21","cited_by_count":103},{"id":"2210.15424","title":"What Language Model to Train if You Have One Million GPU Hours?","created":"2022-10-27","cited_by_count":92},{"id":"2012.11654","title":"Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks","created":"2020-12-21","cited_by_count":87},{"id":"2310.18313","title":"FP8-LM: Training FP8 Large Language Models","created":"2023-10-27","cited_by_count":77},{"id":"2304.13013","title":"Stable and low-precision training for large-scale vision-language models","created":"2023-04-25","cited_by_count":75},{"id":"2404.10102","title":"Chinchilla Scaling: A replication attempt","created":"2024-04-15","cited_by_count":72},{"id":"2309.16620","title":"Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit","created":"2023-09-28","cited_by_count":66},{"id":"2108.10573","title":"The staircase property: How hierarchical structure can guide deep learning","created":"2021-08-24","cited_by_count":62},{"id":"2401.02524","title":"Comprehensive Exploration of Synthetic Data Generation: A Survey","created":"2024-01-04","cited_by_count":60},{"id":"2409.07431","title":"Synthetic continued pretraining","created":"2024-09-11","cited_by_count":60},{"id":"2210.11399","title":"Transcending Scaling Laws with 0.1% Extra Compute","created":"2022-10-20","cited_by_count":57},{"id":"2406.08466","title":"Scaling Laws in Linear Regression: Compute, Parameters, and Data","created":"2024-06-12","cited_by_count":57},{"id":"2305.19268","title":"Intriguing Properties of Quantization at Scale","created":"2023-05-30","cited_by_count":49},{"id":"2311.05640","title":"FinGPT: Large Generative Models for a Small Language","created":"2023-11-03","cited_by_count":49},{"id":"2402.04177","title":"Scaling Laws for Downstream Task Performance of Large Language Models","created":"2024-02-06","cited_by_count":49},{"id":"2110.04369","title":"A Loss Curvature Perspective on Training Instability in Deep Learning","created":"2021-10-08","cited_by_count":40},{"id":"2212.12147","title":"The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes","created":"2022-12-23","cited_by_count":25},{"id":"2406.02528","title":"Scalable MatMul-free Language Modeling","created":"2024-06-04","cited_by_count":25},{"id":"2208.11174","title":"Demystifying the Nvidia Ampere Architecture through Microbenchmarking and Instruction-level Analysis","created":"2022-08-23","cited_by_count":9}],"next_cursor":null}