Entity Recognition from Colloquial Text
Not on the map. Why?
Abstract (arXiv)
Extraction of concepts and entities of interest from non-formal texts such as social media posts and informal communication is an important capability for decision support systems in many domains, including healthcare, customer relationship management, and others. Despite the recent advances in training large language models for a variety of natural language processing tasks, the developed models and techniques have mainly focused on formal texts and do not perform as well on colloquial data, which is characterized by a number of distinct challenges. In our research, we focus on the healthcare domain and investigate the problem of symptom recognition from colloquial texts by designing and evaluating several training strategies for BERT-based model fine-tuning. These strategies are distinguished by the choice of the base model, the training corpora, and application of term perturbations in the training data. The best-performing models trained using these strategies outperform the state-of-the-art specialized symptom recognizer by a large margin. Through a series of experiments, we have found specific patterns of model behavior associated with the training strategies we designed. We present design principles for training strategies for effective entity recognition in colloquial texts based on our findings.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 61 references that are in the map. First posted on arXiv 9 Jan 2024.
Builds on
22 of 61 references.
References · 22
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
- 2019A Survey on Recent Advances in Named Entity Recognition from Deep Learning modelscited by 117
- 2020BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervisioncited by 91
- 2020Rethinking Generalization of Neural Models: A Named Entity Recognition Case Studycited by 35
- 2021RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Modelscited by 28
- 2020Biomedical named entity recognition using BERT in the machine reading comprehension frameworkcited by 22
10 more
- 2023SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)cited by 17
- 2020Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improvecited by 16
- 2022Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretationcited by 12
- 2021Clinical Trial Information Extraction with BERTcited by 7
- 2020COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data modelcited by 6
- 2020Quantification of BERT Diagnosis Generalizability Across Medical Specialties Using Semantic Dataset Distancecited by 6
- 2021Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attackcited by 3
- 2022Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognitioncited by 3
- 2020Understanding patient complaint characteristics using contextual clinical BERT embeddingscited by 2
- 2022Handling and extracting key entities from customer conversations using Speech recognition and Named Entity recognitioncited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.