{"list":"other","total":61,"rows":[{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","created":"2021-10-27","cited_by_count":10109},{"id":"2107.03374","title":"Evaluating Large Language Models Trained on Code","created":"2021-07-07","cited_by_count":8320},{"id":"1602.05629","title":"Communication-Efficient Learning of Deep Networks from Decentralized Data","created":"2016-02-17","cited_by_count":7669},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"1905.10044","title":"BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions","created":"2019-05-24","cited_by_count":2183},{"id":"2311.16502","title":"MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI","created":"2023-11-27","cited_by_count":2080},{"id":"2406.01574","title":"MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark","created":"2024-06-03","cited_by_count":1848},{"id":"1904.08920","title":"Towards VQA Models That Can Read","created":"2019-04-18","cited_by_count":1718},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"2203.10244","title":"ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning","created":"2022-03-19","cited_by_count":1479},{"id":"1504.00325","title":"Microsoft COCO Captions: Data Collection and Evaluation Server","created":"2015-04-01","cited_by_count":1462},{"id":"2203.05482","title":"Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time","created":"2022-03-10","cited_by_count":1396},{"id":"2007.00398","title":"DocVQA: A Dataset for VQA on Document Images","created":"2020-07-01","cited_by_count":1317},{"id":"1910.02054","title":"ZeRO: Memory Optimizations Toward Training Trillion Parameter Models","created":"2019-10-04","cited_by_count":1301},{"id":"2003.00295","title":"Adaptive Federated Optimization","created":"2020-02-29","cited_by_count":1046},{"id":"1909.06335","title":"Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification","created":"2019-09-13","cited_by_count":851},{"id":"1902.01046","title":"Towards Federated Learning at Scale: System Design","created":"2019-02-04","cited_by_count":832},{"id":"2306.01708","title":"TIES-Merging: Resolving Interference When Merging Models","created":"2023-06-02","cited_by_count":818},{"id":"2304.11277","title":"PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel","created":"2023-04-21","cited_by_count":660},{"id":"2104.04473","title":"Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM","created":"2021-04-09","cited_by_count":632},{"id":"1805.09767","title":"Local SGD Converges Fast and Communicates Little","created":"2018-05-24","cited_by_count":605},{"id":"2104.12756","title":"InfographicVQA","created":"2021-04-26","cited_by_count":437},{"id":"2101.06840","title":"ZeRO-Offload: Democratizing Billion-Scale Model Training","created":"2021-01-18","cited_by_count":341},{"id":"2107.06917","title":"A Field Guide to Federated Optimization","created":"2021-07-14","cited_by_count":315},{"id":"2205.05198","title":"Reducing Activation Recomputation in Large Transformer Models","created":"2022-05-10","cited_by_count":302},{"id":"1811.02084","title":"Mesh-TensorFlow: Deep Learning for Supercomputers","created":"2018-11-05","cited_by_count":169},{"id":"2311.08105","title":"DiLoCo: Distributed Low-Communication Training of Language Models","created":"2023-11-14","cited_by_count":121},{"id":"2502.04959","title":"No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces","created":"2025-02-07","cited_by_count":96},{"id":"2111.04877","title":"Papaya: Practical, Private, and Scalable Federated Learning","created":"2021-11-08","cited_by_count":90},{"id":"2203.12533","title":"Pathways: Asynchronous Distributed Dataflow for ML","created":"2022-03-23","cited_by_count":82},{"id":"2301.11913","title":"SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient","created":"2023-01-27","cited_by_count":59},{"id":"2406.16768","title":"WARP: On the Benefits of Weight Averaged Rewarded Policies","created":"2024-06-24","cited_by_count":45},{"id":"1511.05950","title":"Staleness-aware Async-SGD for Distributed Deep Learning","created":"2015-11-18","cited_by_count":43},{"id":"2312.08361","title":"Distributed Inference and Fine-tuning of Large Language Models Over The Internet","created":"2023-12-13","cited_by_count":42},{"id":"2309.08125","title":"Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates","created":"2023-09-15","cited_by_count":40},{"id":"2103.03239","title":"Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices","created":"2021-03-04","cited_by_count":36},{"id":"2501.18512","title":"Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch","created":"2025-01-30","cited_by_count":28},{"id":"2503.09799","title":"Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo","created":"2025-03-12","cited_by_count":27},{"id":"2401.09135","title":"Asynchronous Local-SGD Training for Language Modeling","created":"2024-01-17","cited_by_count":26},{"id":"2507.02119","title":"Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks","created":"2025-07-02","cited_by_count":22},{"id":"1702.05843","title":"Chaos Engineering","created":"2017-02-20","cited_by_count":19},{"id":"2505.23725","title":"MuLoCo: Muon is a practical inner optimizer for DiLoCo","created":"2025-05-29","cited_by_count":18},{"id":"2504.17096","title":"Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters","created":"2025-04-23","cited_by_count":13},{"id":"2508.15706","title":"Overcoming the Communication-Performance Tradeoff in LLM Pretraining","created":"2025-08-21","cited_by_count":12},{"id":"2502.12996","title":"Eager Updates For Overlapped Communication and Computation in DiLoCo","created":"2025-02-18","cited_by_count":11},{"id":"2509.25087","title":"Scaling with Collapse: Efficient and Predictable Training of LLM Families","created":"2025-09-29","cited_by_count":9},{"id":"2506.04531","title":"HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training","created":"2025-06-05","cited_by_count":6},{"id":"2602.00277","title":"Training LLMs with Fault Tolerant HSDP on 100,000 GPUs","created":"2026-01-30","cited_by_count":6},{"id":"2603.08163","title":"Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet","created":"2026-03-09","cited_by_count":6},{"id":"2506.10911","title":"NoLoCo: No-all-reduce Low Communication Training Method for Large Models","created":"2025-06-12","cited_by_count":5},{"id":"2506.21263","title":"DiLoCoX: A Low-Communication Large-Scale Training Framework for Decentralized Cluster","created":"2025-06-26","cited_by_count":4},{"id":"2511.19468","title":"Towards a future space-based, highly scalable AI infrastructure system design","created":"2025-11-22","cited_by_count":4},{"id":"2502.06982","title":"Machine Learning Fleet Efficiency: Analyzing and Optimizing Large-Scale Google TPU Systems with ML Productivity Goodput","created":"2025-02-10","cited_by_count":2},{"id":"2511.13761","title":"What happens when nanochat meets DiLoCo?","created":"2025-11-14","cited_by_count":2},{"id":"2511.17849","title":"Pier: Efficient Large Language Model pretraining with Relaxed Global Communication","created":"2025-11-22","cited_by_count":1},{"id":"2602.04396","title":"LoRDO: Distributed Low-Rank Optimization with Infrequent Communication","created":"2026-02-04","cited_by_count":1}],"next_cursor":null}