# Interpretable Entity Representations through Large-Scale Typing

arXiv: https://arxiv.org/abs/2005.00147 · HTML: https://mlsanity.com/paper/2005.00147/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 19 of 19)

- [CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge](https://mlsanity.com/paper/2109.01653/) (2021)
- [Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model](https://mlsanity.com/paper/2106.04098/) (2021)
- [Modeling Fine-Grained Entity Types with Box Embeddings](https://mlsanity.com/paper/2101.00345/) (2021)
- [Unified Semantic Typing with Meaningful Label Inference](https://mlsanity.com/paper/2205.01826/) (2022)
- [Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference](https://mlsanity.com/paper/2202.06167/) (2022)
- [Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing](https://mlsanity.com/paper/2205.12640/) (2022)
- [Automatic Noisy Label Correction for Fine-Grained Entity Typing](https://mlsanity.com/paper/2205.03011/) (2022)
- [BLINK with Elasticsearch for Efficient Entity Linking in Business Conversations](https://mlsanity.com/paper/2205.04438/) (2022)
- [Instilling Type Knowledge in Language Models via Multi-Task QA](https://mlsanity.com/paper/2204.13796/) (2022)
- [Biomedical Interpretable Entity Representations](https://mlsanity.com/paper/2106.09502/) (2021)
- [Dense Retrieval as Indirect Supervision for Large-space Decision Making](https://mlsanity.com/paper/2310.18619/) (2023)
- [OntoType: Ontology-Guided and Pre-Trained Language Model Assisted Fine-Grained Entity Typing](https://mlsanity.com/paper/2305.12307/) (2023)
- [EnCore: Fine-Grained Entity Typing by Pre-Training Entity Encoders on Coreference Chains](https://mlsanity.com/paper/2305.12924/) (2023)
- [SenteCon: Leveraging Lexicons to Learn Human-Interpretable Language Representations](https://mlsanity.com/paper/2305.14728/) (2023)
- [From Ultra-Fine to Fine: Fine-tuning Ultra-Fine Entity Typing Models to Fine-grained](https://mlsanity.com/paper/2312.06188/) (2023)
- [Intermediate Entity-based Sparse Interpretable Representation Learning](https://mlsanity.com/paper/2212.01641/) (2022)
- [Cross-Lingual Fine-Grained Entity Typing](https://mlsanity.com/paper/2110.07837/) (2021)
- [Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling](https://mlsanity.com/paper/2305.11543/) (2023)
- [Narrative-UFET: Narrative Generation for Ultra-Fine Entity Typing](https://mlsanity.com/paper/2606.27598/) (2026)

## Abstract (arXiv)

In standard methodology for natural language processing, entities in text are typically embedded in dense vector spaces with pre-trained models. The embeddings produced this way are effective when fed into downstream models, but they require end-task fine-tuning and are fundamentally difficult to interpret. In this paper, we present an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box. Our representations are vectors whose values correspond to posterior probabilities over fine-grained entity types, indicating the confidence of a typing model's decision that the entity belongs to the corresponding type. We obtain these representations using a fine-grained entity typing model, trained either on supervised ultra-fine entity typing data (Choi et al. 2018) or distantly-supervised examples from Wikipedia. On entity probing tasks involving recognizing entity identity, our embeddings used in parameter-free downstream models achieve competitive performance with ELMo- and BERT-based embeddings in trained models. We also show that it is possible to reduce the size of our type set in a learning-based way for particular domains. Finally, we show that these embeddings can be post-hoc modified through a small number of rules to incorporate domain knowledge and improve performance.