{"list":"other","total":66,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2103.14030","title":"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows","created":"2021-03-25","cited_by_count":12139},{"id":"1503.02531","title":"Distilling the Knowledge in a Neural Network","created":"2015-03-09","cited_by_count":10324},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"2012.12877","title":"Training data-efficient image transformers & distillation through attention","created":"2020-12-23","cited_by_count":4586},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"1506.02626","title":"Learning both Weights and Connections for Efficient Neural Networks","created":"2015-06-08","cited_by_count":2031},{"id":"1806.09055","title":"DARTS: Differentiable Architecture Search","created":"2018-06-24","cited_by_count":1798},{"id":"1611.01578","title":"Neural Architecture Search with Reinforcement Learning","created":"2016-11-05","cited_by_count":1499},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"2111.09883","title":"Swin Transformer V2: Scaling Up Capacity and Resolution","created":"2021-11-18","cited_by_count":1228},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"1603.05279","title":"XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks","created":"2016-03-16","cited_by_count":849},{"id":"1810.02340","title":"SNIP: Single-shot Network Pruning based on Connection Sensitivity","created":"2018-10-04","cited_by_count":812},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"2309.10305","title":"Baichuan 2: Open Large-scale Language Models","created":"2023-09-19","cited_by_count":670},{"id":"1712.01312","title":"Learning Sparse Neural Networks through $L_0$ Regularization","created":"2017-12-04","cited_by_count":583},{"id":"1608.03665","title":"Learning Structured Sparsity in Deep Neural Networks","created":"2016-08-12","cited_by_count":541},{"id":"1911.11134","title":"Rigging the Lottery: Making All Tickets Winners","created":"2019-11-25","cited_by_count":495},{"id":"2006.05467","title":"Pruning neural networks without any data by iteratively conserving synaptic flow","created":"2020-06-09","cited_by_count":479},{"id":"2002.07376","title":"Picking Winning Tickets Before Training by Preserving Gradient Flow","created":"2020-02-18","cited_by_count":471},{"id":"1609.07061","title":"Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations","created":"2016-09-22","cited_by_count":426},{"id":"1707.04780","title":"Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science","created":"2017-07-15","cited_by_count":328},{"id":"1611.02167","title":"Designing Neural Network Architectures using Reinforcement Learning","created":"2016-11-07","cited_by_count":279},{"id":"1606.05340","title":"Exponential expressivity in deep neural networks through transient chaos","created":"2016-06-16","cited_by_count":261},{"id":"1511.05641","title":"Net2Net: Accelerating Learning via Knowledge Transfer","created":"2015-11-18","cited_by_count":250},{"id":"2006.04647","title":"Neural Architecture Search without Training","created":"2020-06-08","cited_by_count":244},{"id":"2003.02389","title":"Comparing Rewinding and Fine-tuning in Neural Network Pruning","created":"2020-03-05","cited_by_count":240},{"id":"2101.08134","title":"Zero-Cost Proxies for Lightweight NAS","created":"2021-01-20","cited_by_count":199},{"id":"1806.05393","title":"Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks","created":"2018-06-14","cited_by_count":176},{"id":"2110.03215","title":"Towards Continual Knowledge Learning of Language Models","created":"2021-10-07","cited_by_count":174},{"id":"2102.11535","title":"Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective","created":"2021-02-23","cited_by_count":169},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2205.12393","title":"Fine-tuned Language Models are Continual Learners","created":"2022-05-24","cited_by_count":156},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"1711.05136","title":"Deep Rewiring: Training very sparse deep networks","created":"2017-11-14","cited_by_count":139},{"id":"2308.04014","title":"Continual Pre-Training of Large Language Models: How to (re)warm your model?","created":"2023-08-08","cited_by_count":137},{"id":"2202.03829","title":"TimeLMs: Diachronic Language Models from Twitter","created":"2022-02-08","cited_by_count":131},{"id":"1902.08142","title":"Evaluating the Search Phase of Neural Architecture Search","created":"2019-02-21","cited_by_count":120},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110},{"id":"1906.06307","title":"A Signal Propagation Perspective for Pruning Neural Networks at Initialization","created":"2019-06-14","cited_by_count":96},{"id":"2108.04392","title":"Rethinking Architecture Selection in Differentiable NAS","created":"2021-08-10","cited_by_count":92},{"id":"2303.00980","title":"Learning to Grow Pretrained Models for Efficient Transformer Training","created":"2023-03-02","cited_by_count":83},{"id":"2110.07143","title":"bert2BERT: Towards Reusable Pretrained Language Models","created":"2021-10-14","cited_by_count":70},{"id":"2203.06311","title":"ELLE: Efficient Lifelong Pre-training for Emerging Data","created":"2022-03-12","cited_by_count":70},{"id":"2006.09081","title":"Progressive Skeletonization: Trimming more fat from a network at initialization","created":"2020-06-16","cited_by_count":64},{"id":"1710.10348","title":"Multi-level Residual Networks from Dynamical Systems View","created":"2017-10-27","cited_by_count":59},{"id":"2305.09828","title":"Mimetic Initialization of Self-Attention Layers","created":"2023-05-16","cited_by_count":54},{"id":"2011.11778","title":"KeepAugment: A Simple Information-Preserving Data Augmentation Approach","created":"2020-11-23","cited_by_count":52},{"id":"2203.06211","title":"Staged Training for Transformer Language Models","created":"2022-03-11","cited_by_count":49},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45},{"id":"2012.15283","title":"ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning","created":"2020-12-30","cited_by_count":38},{"id":"2105.13880","title":"Knowledge Inheritance for Pre-trained Language Models","created":"2021-05-28","cited_by_count":38},{"id":"2111.13293","title":"KNAS: Green Neural Architecture Search","created":"2021-11-26","cited_by_count":33},{"id":"2011.13635","title":"Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup","created":"2020-11-27","cited_by_count":31},{"id":"2304.02840","title":"NTK-SAP: Improving neural network pruning by aligning training dynamics","created":"2023-04-06","cited_by_count":30},{"id":"2203.09137","title":"Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning","created":"2022-03-17","cited_by_count":22},{"id":"2307.12168","title":"Hallucination Improves the Performance of Unsupervised Visual Representation Learning","created":"2023-07-22","cited_by_count":16},{"id":"2307.14612","title":"GenCo: An Auxiliary Generator from Contrastive Learning for Enhanced Few-Shot Learning in Remote Sensing","created":"2023-07-27","cited_by_count":8}],"next_cursor":null}