# Auto-tagging of Short Conversational Sentences using Transformer Methods

arXiv: https://arxiv.org/abs/2106.01735 · HTML: https://mlsanity.com/paper/2106.01735/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Load What You Need: Smaller Versions of Multilingual BERT](https://mlsanity.com/paper/2010.05609/) (2020)
- [Is Multilingual BERT Fluent in Language Generation?](https://mlsanity.com/paper/1910.03806/) (2019)
- [Using GPT-2 to Create Synthetic Data to Improve the Prediction Performance of NLP Machine Learning Classification Models](https://mlsanity.com/paper/2104.10658/) (2021)

## Built on by (most cited first, 2 of 2)

- [Profitable Trade-Off Between Memory and Performance In Multi-Domain Chatbot Architectures](https://mlsanity.com/paper/2111.03963/) (2021)
- [Training Natural Language Processing Models on Encrypted Text for Enhanced Privacy](https://mlsanity.com/paper/2305.03497/) (2023)

## Abstract (arXiv)

The problem of categorizing short speech sentences according to their semantic features with high accuracy is a subject studied in natural language processing. In this study, a data set created with samples classified in 46 different categories was used. Examples consist of sentences taken from chat conversations between a company's customer representatives and the company's website visitors. The primary purpose is to automatically tag questions and requests from visitors in the most accurate way for 46 predetermined categories for use in a chat application to generate meaningful answers to the questions asked by the website visitors. For this, different BERT models and one GPT-2 model, pre-trained in Turkish, were preferred. The classification performances of the relevant models were analyzed in detail and reported accordingly.