mlsanityData to 7 Oct 2026

Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection

arXiv 2101.09368 ↗First posted 22 Jan 2021cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Lexical semantic change detection is a new and innovative research field. The optimal fine-tuning of models including pre- and post-processing is largely unclear. We optimize existing models by (i) pre-training on large corpora and refining on diachronic target corpora tackling the notorious small data problem, and (ii) applying post-processing transformations that have been shown to improve performance on synchronic tasks. Our results provide a guide for the application and optimization of lexical semantic change detection models across various learning scenarios.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 22 Jan 2021.

Builds on

24 of 54 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  6. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
  7. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 105
  8. 2018Diachronic word embeddings and semantic shifts: a surveycited by 90
  9. 2014Temporal Analysis of Language through Neural Language Modelscited by 53
  10. 2017Simple and Effective Dimensionality Reduction for Word Embeddingscited by 38
  11. 2018Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Changecited by 35
  12. 2019Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Changecited by 32
12 more
  1. 2020UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detectioncited by 32
  2. 2019A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domainscited by 31
  3. 2020RuSemShift: a dataset of historical lexical semantic change in Russiancited by 16
  4. 2021Challenges for Computational Lexical Semantic Changecited by 15
  5. 2018Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluationcited by 8
  6. 2017Representing Sentences as Low-Rank Subspacescited by 7
  7. 2020UWB at SemEval-2020 Task 1: Lexical Semantic Change Detectioncited by 7
  8. 2020IMS at SemEval-2020 Task 1: How low can you go? Dimensionality in Lexical Semantic Change Detectioncited by 5
  9. 2020OP-IMS @ DIACR-Ita: Back to the Roots: SGNS+OP+CD still rocks Semantic Change Detectioncited by 5
  10. 2020CL-IMS @ DIACR-Ita: Volente o Nolente: BERT does not outperform SGNS on Semantic Change Detectioncited by 4
  11. 2019Second-order Co-occurrence Sensitivity of Skip-Gram with Negative Samplingcited by 2
  12. 2020Unsupervised Embedding-based Detection of Lexical Semantic Changescited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 0; 2024: 2.

4 papers.

  1. 2022Linear Adversarial Concept Erasurecited by 98
  2. 2021Lexical Semantic Change Discoverycited by 13
  3. 2024Graph-based Clustering for Detecting Semantic Change Across Time and Languagescited by 10
  4. 2024Presence or Absence: Are Unknown Word Usages in Dictionaries?cited by 3