mlsanityData to 14 Oct 2026

Machine Learning Meets Natural Language Processing -- The story so far

arXiv 2104.10213 ↗First posted 27 Mar 2021cs.CL cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Natural Language Processing (NLP) has evolved significantly over the last decade. This paper highlights the most important milestones of this period while trying to pinpoint the contribution of each individual model and algorithm to the overall progress. Furthermore, it focuses on issues still remaining to be solved, emphasizing the groundbreaking proposals of Transformers, BERT, and all the similar attention-based models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 62 references that are in the map. First posted on arXiv 27 Mar 2021.

Builds on

28 of 62 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  10. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  12. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
16 more
  1. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  2. 2018Deep contextualized word representationscited by 2,290
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  8. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  9. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 302
  10. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 263
  11. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 231
  12. 2016Enhanced LSTM for Natural Language Inferencecited by 200
  13. 2014A Convolutional Neural Network for Modelling Sentencescited by 163
  14. 2017Comparative Study of CNN and RNN for Natural Language Processingcited by 82
  15. 2015A Primer on Neural Network Models for Natural Language Processingcited by 76
  16. 2017Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Contextcited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1.

1 paper.

  1. 2021Automated Essay Scoring Using Transformer Modelscited by 14