{"list":"other","total":67,"rows":[{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"1602.05629","title":"Communication-Efficient Learning of Deep Networks from Decentralized Data","created":"2016-02-17","cited_by_count":7669},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"1912.04977","title":"Advances and Open Problems in Federated Learning","created":"2019-12-10","cited_by_count":3046},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1812.06127","title":"Federated Optimization in Heterogeneous Networks","created":"2018-12-14","cited_by_count":2987},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"1905.10044","title":"BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions","created":"2019-05-24","cited_by_count":2183},{"id":"1809.02789","title":"Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering","created":"2018-09-08","cited_by_count":2164},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1910.02054","title":"ZeRO: Memory Optimizations Toward Training Trillion Parameter Models","created":"2019-10-04","cited_by_count":1301},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"1803.02999","title":"On First-Order Meta-Learning Algorithms","created":"2018-03-08","cited_by_count":1018},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"1902.01046","title":"Towards Federated Learning at Scale: System Design","created":"2019-02-04","cited_by_count":832},{"id":"2306.01708","title":"TIES-Merging: Resolving Interference When Merging Models","created":"2023-06-02","cited_by_count":818},{"id":"2103.16257","title":"Model-Contrastive Federated Learning","created":"2021-03-30","cited_by_count":766},{"id":"2204.06745","title":"GPT-NeoX-20B: An Open-Source Autoregressive Language Model","created":"2022-04-14","cited_by_count":713},{"id":"2104.04473","title":"Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM","created":"2021-04-09","cited_by_count":632},{"id":"1805.09767","title":"Local SGD Converges Fast and Communicates Little","created":"2018-05-24","cited_by_count":605},{"id":"2111.04263","title":"Federated Learning Based on Dynamic Regularization","created":"2021-11-08","cited_by_count":562},{"id":"2101.06840","title":"ZeRO-Offload: Democratizing Billion-Scale Model Training","created":"2021-01-18","cited_by_count":341},{"id":"2107.06917","title":"A Field Guide to Federated Optimization","created":"2021-07-14","cited_by_count":315},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"1802.05799","title":"Horovod: fast and easy distributed deep learning in TensorFlow","created":"2018-02-15","cited_by_count":279},{"id":"1909.12326","title":"Model Pruning Enables Efficient Federated Learning on Edge Devices","created":"2019-09-26","cited_by_count":222},{"id":"1808.07217","title":"Don't Use Large Mini-Batches, Use Local SGD","created":"2018-08-22","cited_by_count":202},{"id":"2010.01243","title":"Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies","created":"2020-10-03","cited_by_count":198},{"id":"1611.04482","title":"Practical Secure Aggregation for Federated Learning on User-Held Data","created":"2016-11-14","cited_by_count":178},{"id":"1812.07210","title":"Expanding the Reach of Federated Learning by Reducing Client Resource Requirements","created":"2018-12-18","cited_by_count":163},{"id":"2309.00363","title":"FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning","created":"2023-09-01","cited_by_count":149},{"id":"2402.06954","title":"OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning","created":"2024-02-10","cited_by_count":128},{"id":"2009.06192","title":"A Principled Approach to Data Valuation for Federated Learning","created":"2020-09-14","cited_by_count":127},{"id":"2311.08105","title":"DiLoCo: Distributed Low-Communication Training of Language Models","created":"2023-11-14","cited_by_count":121},{"id":"2402.15627","title":"MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs","created":"2024-02-23","cited_by_count":112},{"id":"2111.04877","title":"Papaya: Practical, Private, and Scalable Federated Learning","created":"2021-11-08","cited_by_count":90},{"id":"2209.01188","title":"Petals: Collaborative Inference and Fine-tuning of Large Models","created":"2022-09-02","cited_by_count":87},{"id":"1909.08525","title":"Measure Contribution of Participants in Federated Learning","created":"2019-09-17","cited_by_count":79},{"id":"2106.07820","title":"On Large-Cohort Training for Federated Learning","created":"2021-06-15","cited_by_count":74},{"id":"2206.11488","title":"On the Importance and Applicability of Pre-Training for Federated Learning","created":"2022-06-23","cited_by_count":71},{"id":"2210.08090","title":"Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning","created":"2022-10-14","cited_by_count":71},{"id":"2106.04502","title":"Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing","created":"2021-06-08","cited_by_count":58},{"id":"2306.10209","title":"ZeRO++: Extremely Efficient Collective Communication for Giant Model Training","created":"2023-06-16","cited_by_count":52},{"id":"2006.09092","title":"Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training","created":"2020-06-16","cited_by_count":50},{"id":"2403.07648","title":"Characterization of Large Language Model Development in the Datacenter","created":"2024-03-12","cited_by_count":46},{"id":"2312.08361","title":"Distributed Inference and Fine-tuning of Large Language Models Over The Internet","created":"2023-12-13","cited_by_count":42},{"id":"2010.12229","title":"Throughput-Optimal Topology Design for Cross-Silo Federated Learning","created":"2020-10-23","cited_by_count":39},{"id":"2111.14655","title":"FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization","created":"2021-11-29","cited_by_count":31},{"id":"2405.10853","title":"The Future of Large Language Model Pre-training is Federated","created":"2024-05-17","cited_by_count":30},{"id":"2307.09619","title":"Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning","created":"2023-07-18","cited_by_count":29},{"id":"2002.02090","title":"Faster On-Device Training Using New Federated Momentum Algorithm","created":"2020-02-06","cited_by_count":25},{"id":"2110.08133","title":"Trade-offs of Local SGD at Scale: An Empirical Study","created":"2021-10-15","cited_by_count":20},{"id":"2302.12862","title":"FLINT: A Platform for Federated Learning Integration","created":"2023-02-24","cited_by_count":17},{"id":"2405.14791","title":"Recurrent Early Exits for Federated Learning with Heterogeneous Clients","created":"2024-05-23","cited_by_count":12},{"id":"2202.11453","title":"Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization","created":"2022-02-23","cited_by_count":11},{"id":"2409.13155","title":"Convergence of Distributed Adaptive Optimization with Local Updates","created":"2024-09-20","cited_by_count":10},{"id":"2106.10734","title":"Is Shapley Value fair? Improving Client Selection for Mavericks in Federated Learning","created":"2021-06-20","cited_by_count":7},{"id":"2410.12707","title":"FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression","created":"2024-10-16","cited_by_count":7},{"id":"2410.05021","title":"DEPT: Decoupled Embeddings for Pre-training Language Models","created":"2024-10-07","cited_by_count":5},{"id":"2308.16498","title":"Generalised Winograd Schema and its Contextuality","created":"2023-08-31","cited_by_count":4},{"id":"2312.03549","title":"Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment","created":"2023-12-06","cited_by_count":4},{"id":"1810.06877","title":"Collaborative Deep Learning Across Multiple Data Centers","created":"2018-10-16","cited_by_count":3},{"id":"2409.13198","title":"Exploring Scaling Laws for Local SGD in Large Language Model Training","created":"2024-09-20","cited_by_count":2}],"next_cursor":null}