# Term Expansion and FinBERT fine-tuning for Hypernym and Synonym Ranking of Financial Terms

arXiv: https://arxiv.org/abs/2107.13764 · HTML: https://mlsanity.com/paper/2107.13764/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings](https://mlsanity.com/paper/2007.11201/) (2020)
- [FinMatcher at FinSim-2: Hypernym Detection in the Financial Services Domain using Knowledge Graphs](https://mlsanity.com/paper/2103.01576/) (2021)
- [JSI at the FinSim-2 task: Ontology-Augmented Financial Concept Classification](https://mlsanity.com/paper/2106.09230/) (2021)

## Built on by (most cited first, 1 of 1)

- [Learning Semantic Text Similarity to rank Hypernyms of Financial Terms](https://mlsanity.com/paper/2303.13475/) (2023)

## Abstract (arXiv)

Hypernym and synonym matching are one of the mainstream Natural Language Processing (NLP) tasks. In this paper, we present systems that attempt to solve this problem. We designed these systems to participate in the FinSim-3, a shared task of FinNLP workshop at IJCAI-2021. The shared task is focused on solving this problem for the financial domain. We experimented with various transformer based pre-trained embeddings by fine-tuning these for either classification or phrase similarity tasks. We also augmented the provided dataset with abbreviations derived from prospectus provided by the organizers and definitions of the financial terms from DBpedia [Auer et al., 2007], Investopedia, and the Financial Industry Business Ontology (FIBO). Our best performing system uses both FinBERT [Araci, 2019] and data augmentation from the afore-mentioned sources. We observed that term expansion using data augmentation in conjunction with semantic similarity is beneficial for this task and could be useful for the other tasks that deal with short phrases. Our best performing model (Accuracy: 0.917, Rank: 1.156) was developed by fine-tuning SentenceBERT [Reimers et al., 2019] (with FinBERT at the backend) over an extended labelled set created using the hierarchy of labels present in FIBO.