# Semantic Frame Induction using Masked Word Embeddings and Two-Step Clustering

arXiv: https://arxiv.org/abs/2105.13466 · HTML: https://mlsanity.com/paper/2105.13466/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 16 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings](https://mlsanity.com/paper/1905.01739/) (2019)

## Built on by (most cited first, 9 of 9)

- [What is Event Knowledge Graph: A Survey](https://mlsanity.com/paper/2112.15280/) (2021)
- [Unsupervised Slot Schema Induction for Task-oriented Dialog](https://mlsanity.com/paper/2205.04515/) (2022)
- [In the LLM era, Word Sense Induction remains unsolved](https://mlsanity.com/paper/2603.11686/) (2026)
- [Semantic Frame Induction with Deep Metric Learning](https://mlsanity.com/paper/2304.14286/) (2023)
- [Injecting Wiktionary to improve token-level contextual representations using contrastive learning](https://mlsanity.com/paper/2402.07817/) (2024)
- [Acquiring Frame Element Knowledge with Deep Metric Learning for Semantic Frame Induction](https://mlsanity.com/paper/2305.13944/) (2023)
- [CALE : Concept-Aligned Embeddings for Both Within-Lemma and Inter-Lemma Sense Differentiation](https://mlsanity.com/paper/2508.04494/) (2025)
- [FrameEOL: Semantic Frame Induction using Causal Language Models](https://mlsanity.com/paper/2510.09097/) (2025)
- [A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation](https://mlsanity.com/paper/2609.26527/) (2026)

## Abstract (arXiv)

Recent studies on semantic frame induction show that relatively high performance has been achieved by using clustering-based methods with contextualized word embeddings. However, there are two potential drawbacks to these methods: one is that they focus too much on the superficial information of the frame-evoking verb and the other is that they tend to divide the instances of the same verb into too many different frame clusters. To overcome these drawbacks, we propose a semantic frame induction method using masked word embeddings and two-step clustering. Through experiments on the English FrameNet data, we demonstrate that using the masked word embeddings is effective for avoiding too much reliance on the surface information of frame-evoking verbs and that two-step clustering can improve the number of resulting frame clusters for the instances of the same verb.