{"list":"other","total":56,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1911.02116","title":"Unsupervised Cross-lingual Representation Learning at Scale","created":"2019-11-05","cited_by_count":3935},{"id":"1310.4546","title":"Distributed Representations of Words and Phrases and their Compositionality","created":"2013-10-16","cited_by_count":3474},{"id":"1802.05365","title":"Deep contextualized word representations","created":"2018-02-15","cited_by_count":2290},{"id":"1901.08746","title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","created":"2019-01-25","cited_by_count":1947},{"id":"1903.10676","title":"SciBERT: A Pretrained Language Model for Scientific Text","created":"2019-03-26","cited_by_count":1414},{"id":"2112.11446","title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","created":"2021-12-08","cited_by_count":1202},{"id":"1905.05950","title":"BERT Rediscovers the Classical NLP Pipeline","created":"2019-05-15","cited_by_count":1059},{"id":"1906.04341","title":"What Does BERT Look At? An Analysis of BERT's Attention","created":"2019-06-11","cited_by_count":1003},{"id":"1902.01007","title":"Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference","created":"2019-02-04","cited_by_count":876},{"id":"2002.12327","title":"A Primer in BERTology: What we know about how BERT works","created":"2020-02-27","cited_by_count":815},{"id":"1912.00582","title":"BLiMP: The Benchmark of Linguistic Minimal Pairs for English","created":"2019-12-02","cited_by_count":564},{"id":"2003.08271","title":"Pre-trained Models for Natural Language Processing: A Survey","created":"2020-03-18","cited_by_count":505},{"id":"2106.04554","title":"A Survey of Transformers","created":"2021-06-08","cited_by_count":401},{"id":"1611.01368","title":"Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies","created":"2016-11-04","cited_by_count":354},{"id":"1907.13528","title":"What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models","created":"2019-07-31","cited_by_count":320},{"id":"1503.00075","title":"Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks","created":"2015-02-28","cited_by_count":306},{"id":"2002.05867","title":"Transformers as Soft Reasoners over Language","created":"2020-02-14","cited_by_count":270},{"id":"1906.08230","title":"Evaluating Protein Transfer Learning with TAPE","created":"2019-06-19","cited_by_count":231},{"id":"1907.07355","title":"Probing Neural Network Comprehension of Natural Language Arguments","created":"2019-07-17","cited_by_count":214},{"id":"1901.05287","title":"Assessing BERT's Syntactic Abilities","created":"2019-01-16","cited_by_count":192},{"id":"2005.03692","title":"A Systematic Assessment of Syntactic Generalization in Neural Language Models","created":"2020-05-07","cited_by_count":165},{"id":"2007.06225","title":"ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing","created":"2020-07-13","cited_by_count":162},{"id":"2206.13517","title":"ProGen2: Exploring the Boundaries of Protein Language Models","created":"2022-06-27","cited_by_count":159},{"id":"2112.10508","title":"Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP","created":"2021-12-20","cited_by_count":132},{"id":"1902.03249","title":"Insertion Transformer: Flexible Sequence Generation via Insertion Operations","created":"2019-02-08","cited_by_count":125},{"id":"2111.09509","title":"How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN","created":"2021-11-18","cited_by_count":124},{"id":"2006.15222","title":"BERTology Meets Biology: Interpreting Attention in Protein Language Models","created":"2020-06-26","cited_by_count":85},{"id":"2204.13509","title":"On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model","created":"2022-04-28","cited_by_count":81},{"id":"2206.02096","title":"PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding","created":"2022-06-05","cited_by_count":77},{"id":"2001.03632","title":"Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks","created":"2020-01-10","cited_by_count":72},{"id":"1711.09576","title":"Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples","created":"2017-11-27","cited_by_count":66},{"id":"2101.00403","title":"Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words","created":"2021-01-02","cited_by_count":63},{"id":"2012.15671","title":"Vocabulary Learning via Optimal Transport for Neural Machine Translation","created":"2020-12-31","cited_by_count":58},{"id":"1902.02192","title":"Non-Monotonic Sequential Text Generation","created":"2019-02-05","cited_by_count":55},{"id":"2112.07782","title":"Deciphering antibody affinity maturation with language models and weakly supervised learning","created":"2021-12-14","cited_by_count":53},{"id":"1712.03198","title":"Using simulation studies to evaluate statistical methods","created":"2017-12-08","cited_by_count":49},{"id":"2107.00676","title":"A Primer on Pretrained Multilingual Language Models","created":"2021-07-01","cited_by_count":49},{"id":"2005.00633","title":"From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers","created":"2020-05-01","cited_by_count":35},{"id":"2204.04213","title":"Structure-aware Protein Self-supervised Learning","created":"2022-04-06","cited_by_count":33},{"id":"2010.15036","title":"A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models","created":"2020-10-28","cited_by_count":30},{"id":"2109.04953","title":"Does Pretraining for Summarization Require Knowledge Transfer?","created":"2021-09-10","cited_by_count":30},{"id":"2004.00033","title":"Give your Text Representation Models some Love: the Case for Basque","created":"2020-03-31","cited_by_count":28},{"id":"2012.02679","title":"What is a meaningful representation of protein sequences?","created":"2020-11-28","cited_by_count":22},{"id":"1709.10380","title":"An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks","created":"2017-09-29","cited_by_count":21},{"id":"2001.01589","title":"Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation","created":"2020-01-02","cited_by_count":11},{"id":"1809.04179","title":"What can linguistics and deep learning contribute to each other?","created":"2018-09-11","cited_by_count":10},{"id":"2004.09205","title":"A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT","created":"2020-04-20","cited_by_count":8},{"id":"2005.05477","title":"Neural Polysynthetic Language Modelling","created":"2020-05-11","cited_by_count":7},{"id":"2009.13101","title":"Distillation of Weighted Automata from Recurrent Neural Networks using a Spectral Approach","created":"2020-09-28","cited_by_count":6},{"id":"2103.07162","title":"Is BERT a Cross-Disciplinary Knowledge Learner? A Surprising Finding of Pre-trained Models' Transferability","created":"2021-03-12","cited_by_count":6},{"id":"1909.03135","title":"To lemmatize or not to lemmatize: how word normalisation affects ELMo performance in word sense disambiguation","created":"2019-09-06","cited_by_count":4},{"id":"2109.04876","title":"Integrating Approaches to Word Representation","created":"2021-09-10","cited_by_count":4}],"next_cursor":null}