mlsanityData to 7 Oct 2026

Teaching a Massive Open Online Course on Natural Language Processing

arXiv 2104.12846 ↗First posted 26 Apr 2021cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents a new Massive Open Online Course on Natural Language Processing, targeted at non-English speaking students. The course lasts 12 weeks; every week consists of lectures, practical sessions, and quiz assignments. Three weeks out of 12 are followed by Kaggle-style coding assignments. Our course intends to serve multiple purposes: (i) familiarize students with the core concepts and methods in NLP, such as language modeling or word or sentence representations, (ii) show that recent advances, including pre-trained Transformer-based models, are built upon these concepts; (iii) introduce architectures for most demanded real-life applications, (iv) develop practical skills to process texts in multiple languages. The course was prepared and recorded during 2020, launched by the end of the year, and in early 2021 has received positive feedback.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 93 references that are in the map. First posted on arXiv 26 Apr 2021.

Builds on

42 of 93 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  11. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
30 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  8. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  9. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  10. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  11. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  13. 2014Distributed Representations of Sentences and Documentscited by 677
  14. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  15. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  16. 2017Non-Autoregressive Neural Machine Translationcited by 457
  17. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  18. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  19. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  20. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  21. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  22. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  23. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
  24. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  25. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
  26. 2019Entity-Relation Extraction as Multi-Turn Question Answeringcited by 96
  27. 2013Probabilistic Latent Semantic Analysiscited by 63
  28. 2015Morphological Analyzer and Generator for Russian and Ukrainian Languagescited by 25
  29. 2019Reverse-Engineering Satire, or "Paper on Computational Humor Accepted Despite Making Serious Advances"cited by 15
  30. 2019Harry Potter and the Action Prediction Challenge from Natural Languagecited by 2

Built on by

Not cited yet.