mlsanityData to 7 Oct 2026

Computational modeling of semantic change

arXiv 2304.06337 ↗First posted 13 Apr 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

In this chapter we provide an overview of computational modeling for semantic change using large and semi-large textual corpora. We aim to provide a key for the interpretation of relevant methods and evaluation techniques, and also provide insights into important aspects of the computational study of semantic change. We discuss the pros and cons of different classes of models with respect to the properties of the data from which one wishes to model semantic change, and which avenues are available to evaluate the results.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 86 of its 229 references that are in the map. First posted on arXiv 13 Apr 2023.

Builds on

86 of 229 references.

References · 86

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
38 more
  1. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  5. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  8. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  9. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  10. 2020Quantifying Attention Flow in Transformerscited by 759
  11. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  12. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  13. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  14. 2021Causal Abstractions of Neural Networkscited by 401
  15. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  16. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  17. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  18. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  19. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  20. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  21. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  22. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  23. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  24. 2023Localizing Model Behavior with Path Patchingcited by 182
  25. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 170
  26. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  27. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  28. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  29. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  30. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  31. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 123
  32. 2019On Identifiability in Transformerscited by 115
  33. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 105
  34. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  35. 2020Analysing Lexical Semantic Change with Contextualised Word Representationscited by 92
  36. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 92
  37. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 91
  38. 2018Diachronic word embeddings and semantic shifts: a surveycited by 90

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Meaning at the Planck scale? Contextualized word embeddings for doing history, philosophy, and sociology of sciencecited by 3
  2. 2025Dynamic Embedded Topic Models: properties and recommendations based on diverse corporacited by 0