mlsanityData to 7 Oct 2026

Dynamic Contextualized Word Embeddings

arXiv 2010.12684 ↗First posted 23 Oct 2020cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Static word embeddings that represent words by a single vector cannot capture the variability of word meaning in different linguistic and extralinguistic contexts. Building on prior work on contextualized and dynamic word embeddings, we introduce dynamic contextualized word embeddings that represent words as a function of both linguistic and extralinguistic context. Based on a pretrained language model (PLM), dynamic contextualized word embeddings model time and social space jointly, which makes them attractive for a range of NLP tasks involving semantic variability. We highlight potential application scenarios by means of qualitative and quantitative analyses on four English datasets.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 8; 2023: 3; 2024: 3; 2025: 0; 2026: 1.

20 papers.

Sort:

  1. 2022TimeLMs: Diachronic Language Models from Twittercited by 129
  2. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  3. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
  4. 2022Temporal Attention for Language Modelscited by 21
  5. 2023A Survey on Contextualised Semantic Shift Detectioncited by 21
  6. 2022TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Mediacited by 19
  7. 2022Geographic Adaptation of Pretrained Language Modelscited by 14
  8. 2021Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsitycited by 9
  9. 2021LMSOC: An Approach for Socially Sensitive Pretrainingcited by 8
  10. 2024Time Machine GPTcited by 7
  11. 2023PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Promptscited by 3
  12. 2024Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Modelscited by 3
8 more
  1. 2021The challenges of temporal alignment on Twitter during crisescited by 2
  2. 2022Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptationcited by 2
  3. 2022Probabilistic Embeddings with Laplacian Graph Priorscited by 1
  4. 2022Trust in Motion: Capturing Trust Ascendancy in Open-Source Projects using Hybrid AIcited by 0
  5. 2022Domain-Specific Word Embeddings with Structure Predictioncited by 0
  6. 2023WIKITIDE: A Wikipedia-Based Timestamped Definition Pairs Datasetcited by 0
  7. 2024Future Language Modeling from Temporal Document Historycited by 0
  8. 2026Comment-level Topic Drift Analysis in the Reddit Corpuscited by 0