# An Ensemble Approach to Acronym Extraction using Transformers

arXiv: https://arxiv.org/abs/2201.03026 · HTML: https://mlsanity.com/paper/2201.03026/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 9 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networks](https://mlsanity.com/paper/1706.03993/) (2017)
- [AT-BERT: Adversarial Training BERT for Acronym Identification Winning Solution for SDU@AAAI-21](https://mlsanity.com/paper/2101.03700/) (2021)
- [Acronym recognition and processing in 22 languages](https://mlsanity.com/paper/1309.6185/) (2013)

## Abstract (arXiv)

Acronyms are abbreviated units of a phrase constructed by using initial components of the phrase in a text. Automatic extraction of acronyms from a text can help various Natural Language Processing tasks like machine translation, information retrieval, and text summarisation. This paper discusses an ensemble approach for the task of Acronym Extraction, which utilises two different methods to extract acronyms and their corresponding long forms. The first method utilises a multilingual contextual language model and fine-tunes the model to perform the task. The second method relies on a convolutional neural network architecture to extract acronyms and append them to the output of the previous method. We also augment the official training dataset with additional training samples extracted from several open-access journals to help improve the task performance. Our dataset analysis also highlights the noise within the current task dataset. Our approach achieves the following macro-F1 scores on test data released with the task: Danish (0.74), English-Legal (0.72), English-Scientific (0.73), French (0.63), Persian (0.57), Spanish (0.65), Vietnamese (0.65). We release our code and models publicly.