# Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference

arXiv: https://arxiv.org/abs/2202.06167 · HTML: https://mlsanity.com/paper/2202.06167/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 12 Feb 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [A Decomposable Attention Model for Natural Language Inference](https://mlsanity.com/paper/1606.01933/) (2016)
- [Reducing Network Agnostophobia](https://mlsanity.com/paper/1811.04110/) (2018)
- [Ultra-Fine Entity Typing](https://mlsanity.com/paper/1807.04905/) (2018)
- [Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures](https://mlsanity.com/paper/1805.06627/) (2018)
- [Prompt-Learning for Fine-Grained Entity Typing](https://mlsanity.com/paper/2108.10604/) (2021)
- [Can NLI Models Verify QA Systems' Predictions?](https://mlsanity.com/paper/2104.08731/) (2021)
- [Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start](https://mlsanity.com/paper/2010.02584/) (2020)
- [Grounding 'Grounding' in NLP](https://mlsanity.com/paper/2106.02192/) (2021)
- [Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model](https://mlsanity.com/paper/2106.04098/) (2021)
- [Modeling Fine-Grained Entity Types with Box Embeddings](https://mlsanity.com/paper/2101.00345/) (2021)
- [Context-Dependent Fine-Grained Entity Type Tagging](https://mlsanity.com/paper/1412.1820/) (2014)
- [Syntax-Enhanced Pre-trained Model](https://mlsanity.com/paper/2012.14116/) (2020)
- [Learning to Denoise Distantly-Labeled Data for Entity Typing](https://mlsanity.com/paper/1905.01566/) (2019)
- [Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding](https://mlsanity.com/paper/1602.05307/) (2016)
- [Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing](https://mlsanity.com/paper/1903.02591/) (2019)
- [Fine-grained Entity Typing via Label Reasoning](https://mlsanity.com/paper/2109.05744/) (2021)
- [Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss](https://mlsanity.com/paper/1803.03378/) (2018)
- [Interpretable Entity Representations through Large-Scale Typing](https://mlsanity.com/paper/2005.00147/) (2020)
- [Hierarchical Entity Typing via Multi-level Learning to Rank](https://mlsanity.com/paper/2004.02286/) (2020)
- [Improving Fine-grained Entity Typing with Entity Linking](https://mlsanity.com/paper/1909.12079/) (2019)

## Built on by (most cited first, 17 of 17)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [Summarization as Indirect Supervision for Relation Extraction](https://mlsanity.com/paper/2205.09837/) (2022)
- [CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning](https://mlsanity.com/paper/2305.04808/) (2023)
- [Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing](https://mlsanity.com/paper/2205.12640/) (2022)
- [Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?](https://mlsanity.com/paper/2212.10784/) (2022)
- [What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories](https://mlsanity.com/paper/2302.03353/) (2023)
- [Ultra-Fine Entity Typing with Prior Knowledge about Labels: A Simple Clustering Based Strategy](https://mlsanity.com/paper/2305.12802/) (2023)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [OpenStance: Real-world Zero-shot Stance Detection](https://mlsanity.com/paper/2210.14299/) (2022)
- [Learning to Select from Multiple Options](https://mlsanity.com/paper/2212.00301/) (2022)
- [Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field](https://mlsanity.com/paper/2212.01581/) (2022)
- [Recall, Expand and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing](https://mlsanity.com/paper/2212.09125/) (2022)
- [Ontology Enrichment for Effective Fine-grained Entity Typing](https://mlsanity.com/paper/2310.07795/) (2023)
- [Type-enriched Hierarchical Contrastive Strategy for Fine-Grained Entity Typing](https://mlsanity.com/paper/2208.10081/) (2022)
- [OntoType: Ontology-Guided and Pre-Trained Language Model Assisted Fine-Grained Entity Typing](https://mlsanity.com/paper/2305.12307/) (2023)
- [From Ultra-Fine to Fine: Fine-tuning Ultra-Fine Entity Typing Models to Fine-grained](https://mlsanity.com/paper/2312.06188/) (2023)

## Abstract (arXiv)

The task of ultra-fine entity typing (UFET) seeks to predict diverse and free-form words or phrases that describe the appropriate types of entities mentioned in sentences. A key challenge for this task lies in the large amount of types and the scarcity of annotated data per type. Existing systems formulate the task as a multi-way classification problem and train directly or distantly supervised classifiers. This causes two issues: (i) the classifiers do not capture the type semantics since types are often converted into indices; (ii) systems developed in this way are limited to predicting within a pre-defined type set, and often fall short of generalizing to types that are rarely seen or unseen in training. This work presents LITE, a new approach that formulates entity typing as a natural language inference (NLI) problem, making use of (i) the indirect supervision from NLI to infer type information meaningfully represented as textual hypotheses and alleviate the data scarcity issue, as well as (ii) a learning-to-rank objective to avoid the pre-defining of a type set. Experiments show that, with limited training data, LITE obtains state-of-the-art performance on the UFET task. In addition, LITE demonstrates its strong generalizability, by not only yielding best results on other fine-grained entity typing benchmarks, more importantly, a pre-trained LITE system works well on new data containing unseen types.