# Entity Recognition from Colloquial Text

arXiv: https://arxiv.org/abs/2401.04853 · HTML: https://mlsanity.com/paper/2401.04853/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 61 references that are in the map.
- First posted on arXiv 9 Jan 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision](https://mlsanity.com/paper/2006.15509/) (2020)
- [Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study](https://mlsanity.com/paper/2001.03844/) (2020)
- [RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Models](https://mlsanity.com/paper/2109.05620/) (2021)
- [Biomedical named entity recognition using BERT in the machine reading comprehension framework](https://mlsanity.com/paper/2009.01560/) (2020)
- [SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)](https://mlsanity.com/paper/2305.06586/) (2023)
- [Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve](https://mlsanity.com/paper/2004.04564/) (2020)
- [Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretation](https://mlsanity.com/paper/2203.12709/) (2022)
- [Clinical Trial Information Extraction with BERT](https://mlsanity.com/paper/2110.10027/) (2021)
- [COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data model](https://mlsanity.com/paper/2007.10286/) (2020)
- [Quantification of BERT Diagnosis Generalizability Across Medical Specialties Using Semantic Dataset Distance](https://mlsanity.com/paper/2008.06606/) (2020)
- [Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack](https://mlsanity.com/paper/2109.11308/) (2021)
- [Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognition](https://mlsanity.com/paper/2203.10693/) (2022)
- [Understanding patient complaint characteristics using contextual clinical BERT embeddings](https://mlsanity.com/paper/2002.05902/) (2020)
- [Handling and extracting key entities from customer conversations using Speech recognition and Named Entity recognition](https://mlsanity.com/paper/2211.17107/) (2022)

## Built on by (most cited first, 1 of 1)

- [Reliable Decision Support with LLMs: A Framework for Evaluating Consistency in Binary Text Classification Applications](https://mlsanity.com/paper/2505.14918/) (2025)

## Abstract (arXiv)

Extraction of concepts and entities of interest from non-formal texts such as social media posts and informal communication is an important capability for decision support systems in many domains, including healthcare, customer relationship management, and others. Despite the recent advances in training large language models for a variety of natural language processing tasks, the developed models and techniques have mainly focused on formal texts and do not perform as well on colloquial data, which is characterized by a number of distinct challenges. In our research, we focus on the healthcare domain and investigate the problem of symptom recognition from colloquial texts by designing and evaluating several training strategies for BERT-based model fine-tuning. These strategies are distinguished by the choice of the base model, the training corpora, and application of term perturbations in the training data. The best-performing models trained using these strategies outperform the state-of-the-art specialized symptom recognizer by a large margin. Through a series of experiments, we have found specific patterns of model behavior associated with the training strategies we designed. We present design principles for training strategies for effective entity recognition in colloquial texts based on our findings.