# Ultra-Fine Entity Typing with Prior Knowledge about Labels: A Simple Clustering Based Strategy

arXiv: https://arxiv.org/abs/2305.12802 · HTML: https://mlsanity.com/paper/2305.12802/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 58 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://mlsanity.com/paper/2201.05955/) (2022)
- [Adaptive Cross-Modal Few-Shot Learning](https://mlsanity.com/paper/1902.07104/) (2019)
- [Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders](https://mlsanity.com/paper/2104.08027/) (2021)
- [Ultra-Fine Entity Typing](https://mlsanity.com/paper/1807.04905/) (2018)
- [Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network](https://mlsanity.com/paper/2006.05702/) (2020)
- [LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification](https://mlsanity.com/paper/2101.03305/) (2021)
- [Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model](https://mlsanity.com/paper/2106.04098/) (2021)
- [Modeling Fine-Grained Entity Types with Box Embeddings](https://mlsanity.com/paper/2101.00345/) (2021)
- [Context-Dependent Fine-Grained Entity Type Tagging](https://mlsanity.com/paper/1412.1820/) (2014)
- [Syntax-Enhanced Pre-trained Model](https://mlsanity.com/paper/2012.14116/) (2020)
- [Learning to Denoise Distantly-Labeled Data for Entity Typing](https://mlsanity.com/paper/1905.01566/) (2019)
- [Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing](https://mlsanity.com/paper/1903.02591/) (2019)
- [Fine-grained Entity Typing via Label Reasoning](https://mlsanity.com/paper/2109.05744/) (2021)
- [Label Semantics for Few Shot Named Entity Recognition](https://mlsanity.com/paper/2203.08985/) (2022)
- [Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss](https://mlsanity.com/paper/1803.03378/) (2018)
- [Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking](https://mlsanity.com/paper/1807.05127/) (2018)
- [Hierarchical Entity Typing via Multi-level Learning to Rank](https://mlsanity.com/paper/2004.02286/) (2020)
- [Unified Semantic Typing with Meaningful Label Inference](https://mlsanity.com/paper/2205.01826/) (2022)
- [Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference](https://mlsanity.com/paper/2202.06167/) (2022)
- [Neural Architectures for Fine-grained Entity Type Classification](https://mlsanity.com/paper/1606.01341/) (2016)
- [Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention](https://mlsanity.com/paper/2112.01037/) (2021)
- [Automatic Noisy Label Correction for Fine-Grained Entity Typing](https://mlsanity.com/paper/2205.03011/) (2022)
- [Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning](https://mlsanity.com/paper/2105.10195/) (2021)
- [Modelling Commonsense Properties using Pre-Trained Bi-Encoders](https://mlsanity.com/paper/2210.02771/) (2022)
- [Distilling Semantic Concept Embeddings from Contrastively Fine-Tuned Language Models](https://mlsanity.com/paper/2305.09785/) (2023)
- [Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings](https://mlsanity.com/paper/2012.07580/) (2020)

## Built on by (most cited first, 8 of 8)

- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions](https://mlsanity.com/paper/2406.10885/) (2024)
- [All Entities are Not Created Equal: Examining the Long Tail for Ultra-Fine Entity Typing](https://mlsanity.com/paper/2410.17355/) (2024)
- [UltraWiki: Ultra-fine-grained Entity Set Expansion with Negative Seed Entities](https://mlsanity.com/paper/2403.04247/) (2024)
- [What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies](https://mlsanity.com/paper/2310.14793/) (2023)
- [Learning to Correct Noisy Labels for Fine-Grained Entity Typing via Co-Prediction Prompt Tuning](https://mlsanity.com/paper/2310.14596/) (2023)
- [Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings](https://mlsanity.com/paper/2403.16984/) (2024)
- [Narrative-UFET: Narrative Generation for Ultra-Fine Entity Typing](https://mlsanity.com/paper/2606.27598/) (2026)

## Abstract (arXiv)

Ultra-fine entity typing (UFET) is the task of inferring the semantic types, from a large set of fine-grained candidates, that apply to a given entity mention. This task is especially challenging because we only have a small number of training examples for many of the types, even with distant supervision strategies. State-of-the-art models, therefore, have to rely on prior knowledge about the type labels in some way. In this paper, we show that the performance of existing methods can be improved using a simple technique: we use pre-trained label embeddings to cluster the labels into semantic domains and then treat these domains as additional types. We show that this strategy consistently leads to improved results, as long as high-quality label embeddings are used. We furthermore use the label clusters as part of a simple post-processing technique, which results in further performance gains. Both strategies treat the UFET model as a black box and can thus straightforwardly be used to improve a wide range of existing models.