mlsanityData to 7 Oct 2026

Paradigm Shift in Natural Language Processing

arXiv 2109.12575 ↗First posted 26 Sep 2021cs.CL cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

In the era of deep learning, modeling for most NLP tasks has converged to several mainstream paradigms. For example, we usually adopt the sequence labeling paradigm to solve a bundle of tasks such as POS-tagging, NER, Chunking, and adopt the classification paradigm to solve tasks like sentiment analysis. With the rapid progress of pre-trained language models, recent years have observed a rising trend of Paradigm Shift, which is solving one NLP task by reformulating it as another one. Paradigm shift has achieved great success on many tasks, becoming a promising way to improve model performance. Moreover, some of these paradigms have shown great potential to unify a large number of NLP tasks, making it possible to build a single model to handle diverse tasks. In this paper, we review such phenomenon of paradigm shifts in recent years, highlighting several paradigms that have the potential to solve different NLP tasks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 78 of its 105 references that are in the map. First posted on arXiv 26 Sep 2021.

Builds on

78 of 105 references.

References · 78

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
38 more
  1. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  2. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  3. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  4. 2019How Can We Know What Language Models Know?cited by 953
  5. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  6. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  7. 2021GPT Understands, Toocited by 715
  8. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  9. 2016Neural Architectures for Named Entity Recognitioncited by 548
  10. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  11. 2017Non-Autoregressive Neural Machine Translationcited by 457
  12. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  13. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  14. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  15. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  16. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  17. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  18. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  19. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  20. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  21. 2019Depth-Adaptive Transformercited by 227
  22. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  23. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 217
  24. 2016Enhanced LSTM for Natural Language Inferencecited by 199
  25. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
  26. 2021How Many Data Points is a Prompt Worth?cited by 197
  27. 2020Extractive Summarization as Text Matchingcited by 189
  28. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  29. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  30. 2020Event Extraction by Answering (Almost) Natural Questionscited by 177
  31. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  32. 2020Named Entity Recognition as Dependency Parsingcited by 150
  33. 2016Recurrent Neural Network for Text Classification with Multi-Task Learningcited by 144
  34. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  35. 2021Entailment as Few-Shot Learnercited by 122
  36. 2019Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentencecited by 121
  37. 2016Neural Summarization by Extracting Sentences and Wordscited by 114
  38. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 23; 2023: 11; 2024: 2; 2025: 0; 2026: 1.

39 papers.

Sort:

By area:

  1. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  2. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  3. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  4. 2022A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challengescited by 61
  5. 2022BBTv2: Towards a Gradient-Free Future with Large Language Modelscited by 53
  6. 2023MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Predictioncited by 49
  7. 2022Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Trainingcited by 45
  8. 2021Towards Efficient NLP: A Standard Evaluation and A Strong Baselinecited by 44
  9. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44
  10. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  11. 2022FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformerscited by 33
  12. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 32
27 more
  1. 2022A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systemscited by 32
  2. 2022EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Trainingcited by 31
  3. 2021Exploring Universal Intrinsic Task Subspace via Prompt Tuningcited by 25
  4. 2022Late Prompt Tuning: A Late Prompt Could Be Better Than Many Promptscited by 23
  5. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  6. 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 21
  7. 2023Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classificationcited by 17
  8. 2023Generative Knowledge Selection for Knowledge-Grounded Dialoguescited by 15
  9. 2022Masked Vision-Language Transformer in Fashioncited by 14
  10. 2022Constrained Sequence-to-Tree Generation for Hierarchical Text Classificationcited by 13
  11. 2022Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learningcited by 11
  12. 2023Prompt Space Optimizing Few-shot Reasoning Success with Large Language Modelscited by 11
  13. 2022Sequence Generation with Label Augmentation for Relation Extractioncited by 9
  14. 2022Multilingual Normalization of Temporal Expressions with Masked Language Modelscited by 7
  15. 2022Is GPT-3 all you need for Visual Question Answering in Cultural Heritage?cited by 6
  16. 2022A Kind Introduction to Lexical and Grammatical Aspect, with a Survey of Computational Approachescited by 5
  17. 2023Query2Triple: Unified Query Encoding for Answering Diverse Complex Queries over Knowledge Graphscited by 4
  18. 2022Knowledge-informed Molecular Learning: A Survey on Paradigm Transfercited by 3
  19. 2022Mitigating Negative Style Transfer in Hybrid Dialogue Systemcited by 3
  20. 2023Emotion-Cause Pair Extraction as Question Answeringcited by 3
  21. 2022To Answer or Not to Answer? Improving Machine Reading Comprehension Model with Span-based Contrastive Learningcited by 2
  22. 2022Implementation of general formal translatorscited by 1
  23. 2023Analyzing FOMC Minutes: Accuracy and Constraints of Language Modelscited by 1
  24. 2022Changes from Classical Statistics to Modern Statistics and Data Sciencecited by 0
  25. 2023Unveiling the frontiers of deep learning: innovations shaping diverse domainscited by 0
  26. 2024This Candidate is [MASK]. Prompt-based Sentiment Extraction and Reference Letterscited by 0
  27. 2026A Generative Model for Joint Multiple Intent Detection and Slot Fillingcited by 0