# Fine-Tuning Pre-trained Language Models to Detect In-Game Trash Talks

arXiv: https://arxiv.org/abs/2403.15458 · HTML: https://mlsanity.com/paper/2403.15458/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [Evaluating Language Model Finetuning Techniques for Low-resource Languages](https://mlsanity.com/paper/1907.00409/) (2019)
- [Evaluation of GPT and BERT-based models on identifying protein-protein interactions in biomedical text](https://mlsanity.com/paper/2303.17728/) (2023)
- [STFU NOOB! Predicting Crowdsourced Decisions on Toxic Behavior in Online Games](https://mlsanity.com/paper/1404.5905/) (2014)

## Abstract (arXiv)

Common problems in playing online mobile and computer games were related to toxic behavior and abusive communication among players. Based on different reports and studies, the study also discusses the impact of online hate speech and toxicity on players' in-game performance and overall well-being. This study investigates the capability of pre-trained language models to classify or detect trash talk or toxic in-game messages The study employs and evaluates the performance of pre-trained BERT and GPT language models in detecting toxicity within in-game chats. Using publicly available APIs, in-game chat data from DOTA 2 game matches were collected, processed, reviewed, and labeled as non-toxic, mild (toxicity), and toxic. The study was able to collect around two thousand in-game chats to train and test BERT (Base-uncased), BERT (Large-uncased), and GPT-3 models. Based on the three models' state-of-the-art performance, this study concludes pre-trained language models' promising potential for addressing online hate speech and in-game insulting trash talk.