# The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change

arXiv: https://arxiv.org/abs/2311.12664 · HTML: https://mlsanity.com/paper/2311.12664/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Change](https://mlsanity.com/paper/1804.06517/) (2018)
- [DWUG: A large Resource of Diachronic Word Usage Graphs in Four Languages](https://mlsanity.com/paper/2104.08540/) (2021)
- [CoSimLex: A Resource for Evaluating Graded Word Similarity in Context](https://mlsanity.com/paper/1912.05320/) (2019)

## Built on by (most cited first, 7 of 7)

- [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](https://mlsanity.com/paper/2402.12011/) (2024)
- [(Chat)GPT v BERT: Dawn of Justice for Semantic Change Detection](https://mlsanity.com/paper/2401.14040/) (2024)
- [Evaluating Contextualized Representations of (Spanish) Ambiguous Words: A New Lexical Resource and Empirical Analysis](https://mlsanity.com/paper/2406.14678/) (2024)
- [Semantic Change Characterization with LLMs using Rhetorics](https://mlsanity.com/paper/2407.16624/) (2024)
- [The LSCD Benchmark: a Testbed for Diachronic Word Meaning Tasks](https://mlsanity.com/paper/2404.00176/) (2024)
- [Presence or Absence: Are Unknown Word Usages in Dictionaries?](https://mlsanity.com/paper/2406.00656/) (2024)
- [Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity](https://mlsanity.com/paper/2511.21974/) (2025)

## Abstract (arXiv)

We present the DURel tool that implements the annotation of semantic proximity between uses of words into an online, open source interface. The tool supports standardized human annotation as well as computational annotation, building on recent advances with Word-in-Context models. Annotator judgments are clustered with automatic graph clustering techniques and visualized for analysis. This allows to measure word senses with simple and intuitive micro-task judgments between use pairs, requiring minimal preparation efforts. The tool offers additional functionalities to compare the agreement between annotators to guarantee the inter-subjectivity of the obtained judgments and to calculate summary statistics giving insights into sense frequency distributions, semantic variation or changes of senses over time.