# BERT-based Acronym Disambiguation with Multiple Training Strategies

arXiv: https://arxiv.org/abs/2103.00488 · HTML: https://mlsanity.com/paper/2103.00488/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 25 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://mlsanity.com/paper/1605.07725/) (2016)
- [Data Noising as Smoothing in Neural Network Language Models](https://mlsanity.com/paper/1703.02573/) (2017)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation](https://mlsanity.com/paper/2010.14678/) (2020)
- [Deep Contextualized Biomedical Abbreviation Expansion](https://mlsanity.com/paper/1906.03360/) (2019)
- [Acronym Identification and Disambiguation Shared Tasks for Scientific Document Understanding](https://mlsanity.com/paper/2012.11760/) (2020)

## Built on by (most cited first, 8 of 8)

- [LingYi: Medical Conversational Question Answering System based on Multi-modal Knowledge Graphs](https://mlsanity.com/paper/2204.09220/) (2022)
- [GLADIS: A General and Large Acronym Disambiguation Benchmark](https://mlsanity.com/paper/2302.01860/) (2023)
- [SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation](https://mlsanity.com/paper/2111.14306/) (2021)
- [ADBCMM : Acronym Disambiguation by Building Counterfactuals and Multilingual Mixing](https://mlsanity.com/paper/2112.08991/) (2021)
- [Detecting Generated Scientific Papers using an Ensemble of Transformer Models](https://mlsanity.com/paper/2209.08283/) (2022)
- [ABB-BERT: A BERT model for disambiguating abbreviations and contractions](https://mlsanity.com/paper/2207.04008/) (2022)
- [Automated Extraction of Acronym-Expansion Pairs from Scientific Papers](https://mlsanity.com/paper/2412.01093/) (2024)
- [DACE For Railway Acronym Disambiguation](https://mlsanity.com/paper/2512.18357/) (2025)

## Abstract (arXiv)

Acronym disambiguation (AD) task aims to find the correct expansions of an ambiguous ancronym in a given sentence. Although it is convenient to use acronyms, sometimes they could be difficult to understand. Identifying the appropriate expansions of an acronym is a practical task in natural language processing. Since few works have been done for AD in scientific field, we propose a binary classification model incorporating BERT and several training strategies including dynamic negative sample selection, task adaptive pretraining, adversarial training and pseudo labeling in this paper. Experiments on SciAD show the effectiveness of our proposed model and our score ranks 1st in SDU@AAAI-21 shared task 2: Acronym Disambiguation.