{"list":"other","total":66,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2310.06825","title":"Mistral 7B","created":"2023-10-10","cited_by_count":3164},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"1905.10044","title":"BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions","created":"2019-05-24","cited_by_count":2183},{"id":"1809.02789","title":"Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering","created":"2018-09-08","cited_by_count":2164},{"id":"1703.04730","title":"Understanding Black-box Predictions via Influence Functions","created":"2017-03-14","cited_by_count":1741},{"id":"1910.02054","title":"ZeRO: Memory Optimizations Toward Training Trillion Parameter Models","created":"2019-10-04","cited_by_count":1301},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"1604.06174","title":"Training Deep Nets with Sublinear Memory Cost","created":"2016-04-21","cited_by_count":848},{"id":"1707.06209","title":"Crowdsourcing Multiple Choice Science Questions","created":"2017-07-19","cited_by_count":765},{"id":"2401.02954","title":"DeepSeek LLM: Scaling Open-Source Language Models with Longtermism","created":"2024-01-05","cited_by_count":642},{"id":"2402.00838","title":"OLMo: Accelerating the Science of Language Models","created":"2024-02-01","cited_by_count":613},{"id":"2402.04333","title":"LESS: Selecting Influential Data for Targeted Instruction Tuning","created":"2024-02-06","cited_by_count":532},{"id":"2206.14486","title":"Beyond neural scaling laws: beating power law scaling via data pruning","created":"2022-06-29","cited_by_count":528},{"id":"2304.15004","title":"Are Emergent Abilities of Large Language Models a Mirage?","created":"2023-04-28","cited_by_count":523},{"id":"2310.06694","title":"Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning","created":"2023-10-10","cited_by_count":415},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403},{"id":"2305.10429","title":"DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining","created":"2023-05-17","cited_by_count":399},{"id":"2406.11794","title":"DataComp-LM: In search of the next generation of training sets for language models","created":"2024-06-17","cited_by_count":398},{"id":"2302.03169","title":"Data Selection for Language Models via Importance Resampling","created":"2023-02-06","cited_by_count":329},{"id":"2308.03296","title":"Studying Large Language Model Generalization with Influence Functions","created":"2023-08-07","cited_by_count":326},{"id":"2106.07139","title":"Pre-Trained Models: Past, Present and Future","created":"2021-06-14","cited_by_count":319},{"id":"2303.09540","title":"SemDeDup: Data-efficient learning at web-scale through semantic deduplication","created":"2023-03-16","cited_by_count":303},{"id":"1605.07723","title":"Data Programming: Creating Large Training Sets, Quickly","created":"2016-05-25","cited_by_count":246},{"id":"2206.07137","title":"Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt","created":"2022-06-14","cited_by_count":236},{"id":"2402.16827","title":"A Survey on Data Selection for Language Models","created":"2024-02-26","cited_by_count":236},{"id":"2109.04332","title":"PPT: Pre-trained Prompt Tuning for Few-shot Learning","created":"2021-09-09","cited_by_count":232},{"id":"2205.10770","title":"Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models","created":"2022-05-22","cited_by_count":220},{"id":"1905.00877","title":"You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle","created":"2019-05-02","cited_by_count":183},{"id":"2112.10684","title":"Efficient Large Scale Language Modeling with Mixtures of Experts","created":"2021-12-20","cited_by_count":175},{"id":"2308.12284","title":"D4: Improving LLM Pretraining via Document De-Duplication and Diversification","created":"2023-08-23","cited_by_count":150},{"id":"2404.07965","title":"Rho-1: Not All Tokens Are What You Need","created":"2024-04-11","cited_by_count":133},{"id":"2307.14430","title":"Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models","created":"2023-07-26","cited_by_count":121},{"id":"2311.15653","title":"MoDS: Model-oriented Data Selection for Instruction Tuning","created":"2023-11-27","cited_by_count":107},{"id":"2401.12926","title":"DsDm: Model-Aware Dataset Selection with Datamodels","created":"2024-01-23","cited_by_count":105},{"id":"2406.06046","title":"MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models","created":"2024-06-10","cited_by_count":100},{"id":"2310.15393","title":"DoGE: Domain Reweighting with Generalization Estimation","created":"2023-10-23","cited_by_count":90},{"id":"2406.11011","title":"Data Shapley in One Training Run","created":"2024-06-16","cited_by_count":81},{"id":"1710.09513","title":"Maximum Principle Based Algorithms for Deep Learning","created":"2017-10-26","cited_by_count":78},{"id":"2312.02406","title":"Efficient Online Data Mixing For Language Model Pre-Training","created":"2023-12-05","cited_by_count":77},{"id":"1807.01083","title":"A Mean-Field Optimal Control Formulation of Deep Learning","created":"2018-07-03","cited_by_count":67},{"id":"2009.11243","title":"Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves","created":"2020-09-23","cited_by_count":58},{"id":"2305.09137","title":"Pre-Training to Learn in Context","created":"2023-05-16","cited_by_count":42},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41},{"id":"2311.00913","title":"Self-Influence Guided Data Reweighting for Language Model Pre-training","created":"2023-11-02","cited_by_count":37},{"id":"1904.05657","title":"Deep learning as optimal control problems: models and numerical methods","created":"2019-04-11","cited_by_count":34},{"id":"1908.10920","title":"Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective","created":"2019-08-28","cited_by_count":27},{"id":"1803.01299","title":"An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks","created":"2018-03-04","cited_by_count":26},{"id":"2310.15389","title":"Irreducible Curriculum for Language Model Pretraining","created":"2023-10-23","cited_by_count":16},{"id":"2210.09175","title":"Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization","created":"2022-10-17","cited_by_count":15},{"id":"2202.04097","title":"Turnpike in optimal control of PDEs, ResNets, and beyond","created":"2022-02-08","cited_by_count":11},{"id":"2311.11973","title":"Adaptive Training Distributions with Scalable Online Bilevel Optimization","created":"2023-11-20","cited_by_count":11},{"id":"2402.17759","title":"Towards Optimal Learning of Language Models","created":"2024-02-27","cited_by_count":7}],"next_cursor":null}