mlsanityData to 7 Oct 2026

Defining a New NLP Playground

arXiv 2310.20633 ↗First posted 31 Oct 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

The recent explosion of performance of large language models (LLMs) has changed the field of Natural Language Processing (NLP) more abruptly and seismically than any other shift in the field's 80-year history. This has resulted in concerns that the field will become homogenized and resource-intensive. The new status quo has put many academic researchers, especially PhD students, at a disadvantage. This paper aims to define a new NLP playground by proposing 20+ PhD-dissertation-worthy research directions, covering theoretical analysis, new and challenging problems, learning paradigms, and interdisciplinary applications.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 118 of its 186 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

118 of 186 references.

References · 118

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
38 more
  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  13. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  14. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  15. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  16. 2022Mass-Editing Memory in a Transformercited by 976
  17. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  18. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  19. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  20. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  21. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  22. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  23. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  24. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  25. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  26. 2022Galactica: A Large Language Model for Sciencecited by 736
  27. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  28. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  29. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  30. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  31. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  32. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  33. 2021Editing Factual Knowledge in Language Modelscited by 548
  34. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  35. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  36. 2022Memory-Based Model Editing at Scalecited by 443
  37. 2023Larger language models do in-context learning differentlycited by 386
  38. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356

50 most cited of 118 All 118 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 4.

4 papers.

  1. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  2. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 24
  3. 2024FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guaranteescited by 6
  4. 2024Transforming Scholarly Landscapes: Influence of Large Language Models on Academic Fields beyond Computer Sciencecited by 4