{"list":"other","total":60,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2304.08485","title":"Visual Instruction Tuning","created":"2023-04-17","cited_by_count":7869},{"id":"1911.05722","title":"Momentum Contrast for Unsupervised Visual Representation Learning","created":"2019-11-13","cited_by_count":7513},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2301.12597","title":"BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models","created":"2023-01-30","cited_by_count":6137},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1701.06538","title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer","created":"2017-01-23","cited_by_count":3509},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2101.03961","title":"Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity","created":"2021-01-11","cited_by_count":3073},{"id":"1903.12261","title":"Benchmarking Neural Network Robustness to Common Corruptions and Perturbations","created":"2019-03-28","cited_by_count":2877},{"id":"2106.08254","title":"BEiT: BERT Pre-Training of Image Transformers","created":"2021-06-15","cited_by_count":2375},{"id":"2006.16241","title":"The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization","created":"2020-06-29","cited_by_count":1850},{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631},{"id":"2106.01345","title":"Decision Transformer: Reinforcement Learning via Sequence Modeling","created":"2021-06-02","cited_by_count":1546},{"id":"1902.10811","title":"Do ImageNet Classifiers Generalize to ImageNet?","created":"2019-02-13","cited_by_count":1515},{"id":"1907.07174","title":"Natural Adversarial Examples","created":"2019-07-16","cited_by_count":1408},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"2112.06905","title":"GLaM: Efficient Scaling of Language Models with Mixture-of-Experts","created":"2021-12-13","cited_by_count":897},{"id":"2106.05974","title":"Scaling Vision with Sparse Mixture of Experts","created":"2021-06-10","cited_by_count":747},{"id":"2106.02039","title":"Offline Reinforcement Learning as One Big Sequence Modeling Problem","created":"2021-06-03","cited_by_count":644},{"id":"2202.09368","title":"Mixture-of-Experts with Expert Choice Routing","created":"2022-02-18","cited_by_count":623},{"id":"1703.11008","title":"Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data","created":"2017-03-31","cited_by_count":533},{"id":"1912.02178","title":"Fantastic Generalization Measures and Where to Find Them","created":"2019-12-04","cited_by_count":513},{"id":"2111.02358","title":"VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts","created":"2021-11-03","cited_by_count":425},{"id":"2202.08906","title":"ST-MoE: Designing Stable and Transferable Sparse Expert Models","created":"2022-02-17","cited_by_count":399},{"id":"2103.16716","title":"BASE Layers: Simplifying Training of Large, Sparse Models","created":"2021-03-30","cited_by_count":320},{"id":"2202.01169","title":"Unified Scaling Laws for Routed Language Models","created":"2022-02-02","cited_by_count":253},{"id":"2308.00951","title":"From Sparse to Soft Mixtures of Experts","created":"2023-08-02","cited_by_count":248},{"id":"2312.00785","title":"Sequential Modeling Enables Scalable Learning for Large Vision Models","created":"2023-12-01","cited_by_count":212},{"id":"2106.07411","title":"Partial success in closing the gap between human and machine vision","created":"2021-06-14","cited_by_count":185},{"id":"2204.09179","title":"On the Representation Collapse of Sparse Mixture of Experts","created":"2022-04-20","cited_by_count":157},{"id":"2306.00258","title":"Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior","created":"2023-06-01","cited_by_count":131},{"id":"1712.07628","title":"Improving Generalization Performance by Switching from Adam to SGD","created":"2017-12-20","cited_by_count":125},{"id":"2110.04260","title":"Taming Sparsely Activated Transformer with Stochastic Experts","created":"2021-10-08","cited_by_count":113},{"id":"1705.02498","title":"A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions","created":"2017-05-06","cited_by_count":108},{"id":"2304.07993","title":"In-Context Operator Learning with Data Prompts for Differential Equation Problems","created":"2023-04-17","cited_by_count":101},{"id":"2204.08396","title":"StableMoE: Stable Routing Strategy for Mixture of Experts","created":"2022-04-18","cited_by_count":100},{"id":"2206.04046","title":"Sparse Mixture-of-Experts are Domain Generalizable Learners","created":"2022-06-08","cited_by_count":85},{"id":"1807.04740","title":"Negative Momentum for Improved Game Dynamics","created":"2018-07-12","cited_by_count":60},{"id":"2304.09871","title":"A Theory on Adam Instability in Large-Scale Machine Learning","created":"2023-04-19","cited_by_count":56},{"id":"1911.03864","title":"Improving Transformer Models by Reordering their Sublayers","created":"2019-11-10","cited_by_count":52},{"id":"2204.07689","title":"Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners","created":"2022-04-16","cited_by_count":51},{"id":"1904.12774","title":"Routing Networks and the Challenges of Modular and Compositional Computation","created":"2019-04-29","cited_by_count":50},{"id":"2110.04840","title":"Heavy Ball Neural Ordinary Differential Equations","created":"2021-10-10","cited_by_count":42},{"id":"2306.00008","title":"Brainformers: Trading Simplicity for Efficiency","created":"2023-05-29","cited_by_count":31},{"id":"2308.10110","title":"Robust Mixture-of-Expert Training for Convolutional Neural Networks","created":"2023-08-19","cited_by_count":30},{"id":"2210.10253","title":"On the Adversarial Robustness of Mixture of Experts","created":"2022-10-19","cited_by_count":28},{"id":"2002.10583","title":"Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent","created":"2020-02-24","cited_by_count":24},{"id":"2112.05820","title":"Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition","created":"2021-12-10","cited_by_count":16},{"id":"1710.04753","title":"A Robust Accelerated Optimization Algorithm for Strongly Convex Functions","created":"2017-10-13","cited_by_count":14},{"id":"1810.01638","title":"Optimization Algorithm Inspired Deep Neural Network Structure Design","created":"2018-10-03","cited_by_count":14},{"id":"2102.08431","title":"Complex Momentum for Optimization in Games","created":"2021-02-16","cited_by_count":11},{"id":"2110.07034","title":"How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies","created":"2021-10-13","cited_by_count":10},{"id":"1706.01338","title":"Understanding the Learned Iterative Soft Thresholding Algorithm with matrix factorization","created":"2017-06-02","cited_by_count":9},{"id":"2208.00579","title":"Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization","created":"2022-08-01","cited_by_count":4}],"next_cursor":null}