mlsanityData to 7 Oct 2026

Natural Language Processing RELIES on Linguistics

arXiv 2405.05966 ↗First posted 9 May 2024cs.CL cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have become capable of generating highly fluent text in certain languages, without modules specially designed to capture grammar or semantic coherence. What does this mean for the future of linguistic expertise in NLP? We highlight several aspects in which NLP (still) relies on linguistics, or where linguistic thinking can illuminate new directions. We argue our case around the acronym RELIES that encapsulates six major facets where linguistics contributes to NLP: Resources, Evaluation, Low-resource settings, Interpretability, Explanation, and the Study of language. This list is not exhaustive, nor is linguistics the main point of reference for every effort under these themes; but at a macro level, these facets highlight the enduring importance of studying machine systems vis-`a-vis systems of human language.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 99 of its 297 references that are in the map. First posted on arXiv 9 May 2024.

Builds on

99 of 297 references.

References · 99

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2018Datasheets for Datasetscited by 1,371
  8. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  9. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  10. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  11. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
38 more
  1. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  2. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  3. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  4. 2018Gender Bias in Coreference Resolutioncited by 438
  5. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  6. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  7. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  8. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  9. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  10. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  11. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  12. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  13. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  14. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  15. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  16. 2021Thinking Like Transformerscited by 182
  17. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  18. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 169
  19. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  20. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 137
  21. 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 134
  22. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  23. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  24. 2020BLEU might be Guilty but References are not Innocentcited by 109
  25. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  26. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  27. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  28. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  29. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  30. 2023A Benchmark for Learning to Translate a New Language from One Grammar Bookcited by 74
  31. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  32. 2021Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?cited by 55
  33. 2020Leveraging Abstract Meaning Representation for Knowledge Base Question Answeringcited by 53
  34. 2022Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selectioncited by 52
  35. 2024What does the Knowledge Neuron Thesis Have to do with Knowledge?cited by 49
  36. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  37. 2023Language acquisition: do children and language models follow similar learning stages?cited by 40
  38. 2024MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modelingcited by 40

50 most cited of 99 All 99 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 3.

9 papers.

  1. 2025MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairscited by 65
  2. 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challengescited by 31
  3. 2024Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?cited by 23
  4. 2025UD-English-CHILDES: A Collected Resource of Gold and Silver Universal Dependencies Trees for Child Language Interactionscited by 3
  5. 2024Learning from Impairment: Leveraging Insights from Clinical Linguistics in Language Modelling Researchcited by 1
  6. 2026Speaking of Language: Reflections on Metalanguage Research in NLPcited by 1
  7. 2025Using Machine Learning to Detect Fraudulent SMSs in Chichewacited by 0
  8. 2026Comparing Natural and Synthetic Structured Data: A Study of the Passive Verb Alternation in French and Italiancited by 0
  9. 2026Language as an Independent Information Layer: A Conceptual Model of Communication, Cognition and Decision-Makingcited by 0