{"list":"other","total":59,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"1902.00751","title":"Parameter-Efficient Transfer Learning for NLP","created":"2019-02-02","cited_by_count":4236},{"id":"1909.11942","title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","created":"2019-09-26","cited_by_count":2790},{"id":"2106.01946","title":"Convex optimization","created":"2021-06-03","cited_by_count":2515},{"id":"1802.05365","title":"Deep contextualized word representations","created":"2018-02-15","cited_by_count":2290},{"id":"1905.10044","title":"BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions","created":"2019-05-24","cited_by_count":2183},{"id":"1905.00537","title":"SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems","created":"2019-05-02","cited_by_count":1729},{"id":"2104.02057","title":"An Empirical Study of Training Self-Supervised Vision Transformers","created":"2021-04-05","cited_by_count":1616},{"id":"2004.10964","title":"Don't Stop Pretraining: Adapt Language Models to Domains and Tasks","created":"2020-04-23","cited_by_count":1565},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"2102.09690","title":"Calibrate Before Use: Improving Few-Shot Performance of Language Models","created":"2021-02-19","cited_by_count":1392},{"id":"2106.10199","title":"BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models","created":"2021-06-18","cited_by_count":1245},{"id":"1805.12471","title":"Neural Network Acceptability Judgments","created":"2018-05-31","cited_by_count":958},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"2202.10054","title":"Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution","created":"2022-02-21","cited_by_count":739},{"id":"2110.07602","title":"P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks","created":"2021-10-14","cited_by_count":689},{"id":"1912.02292","title":"Deep Double Descent: Where Bigger Models and More Data Hurt","created":"2019-12-04","cited_by_count":672},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1509.01240","title":"Train faster, generalize better: Stability of stochastic gradient descent","created":"2015-09-03","cited_by_count":552},{"id":"2102.00554","title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","created":"2021-01-31","cited_by_count":527},{"id":"2002.06305","title":"Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping","created":"2020-02-15","cited_by_count":384},{"id":"1901.09960","title":"Using Pre-Training Can Improve Model Robustness and Uncertainty","created":"2019-01-28","cited_by_count":373},{"id":"1808.09121","title":"WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations","created":"2018-08-28","cited_by_count":372},{"id":"1802.05296","title":"Stronger generalization bounds for deep nets via a compression approach","created":"2018-02-14","cited_by_count":307},{"id":"1909.11764","title":"FreeLB: Enhanced Adversarial Training for Natural Language Understanding","created":"2019-09-25","cited_by_count":264},{"id":"2006.04884","title":"On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines","created":"2020-06-08","cited_by_count":255},{"id":"2006.05987","title":"Revisiting Few-sample BERT Fine-tuning","created":"2020-06-10","cited_by_count":238},{"id":"1811.01088","title":"Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks","created":"2018-11-02","cited_by_count":215},{"id":"2111.09839","title":"Training Neural Networks with Fixed Sparse Masks","created":"2021-11-18","cited_by_count":214},{"id":"2101.11038","title":"Muppet: Massive Multi-task Representations with Pre-Finetuning","created":"2021-01-26","cited_by_count":192},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1903.05987","title":"To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks","created":"2019-03-14","cited_by_count":167},{"id":"1712.00673","title":"Towards Robust Neural Networks via Random Self-ensemble","created":"2017-12-02","cited_by_count":159},{"id":"2008.03156","title":"Better Fine-Tuning by Reducing Representational Collapse","created":"2020-08-06","cited_by_count":157},{"id":"2106.03164","title":"On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation","created":"2021-06-06","cited_by_count":147},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"2109.05687","title":"Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning","created":"2021-09-13","cited_by_count":146},{"id":"1909.11299","title":"Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models","created":"2019-09-25","cited_by_count":141},{"id":"2211.15583","title":"On the Effectiveness of Parameter-Efficient Fine-Tuning","created":"2022-11-28","cited_by_count":141},{"id":"2005.00628","title":"Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?","created":"2020-05-01","cited_by_count":127},{"id":"2006.08157","title":"Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent","created":"2020-06-15","cited_by_count":119},{"id":"1710.08402","title":"Stability and Generalization of Learning Algorithms that Converge to Global Optima","created":"2017-10-23","cited_by_count":100},{"id":"1703.01678","title":"Data-Dependent Stability of Stochastic Gradient Descent","created":"2017-03-05","cited_by_count":92},{"id":"2106.09226","title":"Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning","created":"2021-06-17","cited_by_count":82},{"id":"2211.05729","title":"How Does Sharpness-Aware Minimization Minimize Sharpness?","created":"2022-11-10","cited_by_count":65},{"id":"1705.05933","title":"Sub-sampled Cubic Regularization for Non-convex Optimization","created":"2017-05-16","cited_by_count":63},{"id":"2002.08709","title":"Do We Need Zero Training Loss After Achieving Zero Training Error?","created":"2020-02-20","cited_by_count":61},{"id":"1906.04659","title":"Stable Rank Normalization for Improved Generalization in Neural Networks and GANs","created":"2019-06-11","cited_by_count":41},{"id":"2004.14129","title":"How fine can fine-tuning be? Learning efficient language models","created":"2020-04-24","cited_by_count":41},{"id":"2003.02249","title":"jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models","created":"2020-03-04","cited_by_count":39},{"id":"2204.07675","title":"MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation","created":"2022-04-15","cited_by_count":37},{"id":"2108.02340","title":"Robust Transfer Learning with Pretrained Language Models through Adapters","created":"2021-08-05","cited_by_count":35},{"id":"2107.04835","title":"Noise Stability Regularization for Improving BERT Fine-tuning","created":"2021-07-10","cited_by_count":29},{"id":"2202.13441","title":"Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond","created":"2022-02-27","cited_by_count":29},{"id":"2207.09150","title":"On the Usability of Transformers-based models for a French Question-Answering task","created":"2022-07-19","cited_by_count":8},{"id":"2210.10325","title":"Improving Stability of Fine-Tuning Pretrained Language Models via Component-Wise Gradient Norm Clipping","created":"2022-10-19","cited_by_count":5}],"next_cursor":null}