# Evaluating Pretrained Transformer Models for Entity Linking in Task-Oriented Dialog

arXiv: https://arxiv.org/abs/2112.08327 · HTML: https://mlsanity.com/paper/2112.08327/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering](https://mlsanity.com/paper/2007.15207/) (2020)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [Rasa: Open Source Language Understanding and Dialogue Management](https://mlsanity.com/paper/1712.05181/) (2017)
- [Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking](https://mlsanity.com/paper/2003.05473/) (2020)
- [NeuSpell: A Neural Spelling Correction Toolkit](https://mlsanity.com/paper/2010.11085/) (2020)
- [What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation](https://mlsanity.com/paper/2010.14678/) (2020)
- [Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks](https://mlsanity.com/paper/1604.00734/) (2016)

## Built on by (most cited first, 3 of 3)

- [Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models](https://mlsanity.com/paper/2309.14735/) (2023)
- [Implementing NLPs in industrial process modeling: Addressing Categorical Variables](https://mlsanity.com/paper/2409.19097/) (2024)
- [InfoTech Assistant: A Multimodal Conversational Agent for InfoTechnology Web Portal Queries](https://mlsanity.com/paper/2412.16412/) (2024)

## Abstract (arXiv)

The wide applicability of pretrained transformer models (PTMs) for natural language tasks is well demonstrated, but their ability to comprehend short phrases of text is less explored. To this end, we evaluate different PTMs from the lens of unsupervised Entity Linking in task-oriented dialog across 5 characteristics -- syntactic, semantic, short-forms, numeric and phonetic. Our results demonstrate that several of the PTMs produce sub-par results when compared to traditional techniques, albeit competitive to other neural baselines. We find that some of their shortcomings can be addressed by using PTMs fine-tuned for text-similarity tasks, which illustrate an improved ability in comprehending semantic and syntactic correspondences, as well as some improvements for short-forms, numeric and phonetic variations in entity mentions. We perform qualitative analysis to understand nuances in their predictions and discuss scope for further improvements. Code can be found at https://github.com/murali1996/el_tod