# bitsa_nlp@LT-EDI-ACL2022: Leveraging Pretrained Language Models for Detecting Homophobia and Transphobia in Social Media Comments

arXiv: https://arxiv.org/abs/2203.14267 · HTML: https://mlsanity.com/paper/2203.14267/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 27 Mar 2022.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Dataset for Identification of Homophobia and Transophobia in Multilingual YouTube Comments](https://mlsanity.com/paper/2109.00227/) (2021)
- [Hate-Alert@DravidianLangTech-EACL2021: Ensembling strategies for Transformer-based Offensive language Detection](https://mlsanity.com/paper/2102.10084/) (2021)
- [Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts](https://mlsanity.com/paper/2108.10939/) (2021)
- [IIITG-ADBU@HASOC-Dravidian-CodeMix-FIRE2020: Offensive Content Detection in Code-Mixed Dravidian Text](https://mlsanity.com/paper/2107.14336/) (2021)
- [Hopeful_Men@LT-EDI-EACL2021: Hope Speech Detection Using Indic Transliteration and Transformers](https://mlsanity.com/paper/2102.12082/) (2021)

## Built on by (most cited first, 3 of 3)

- [A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing](https://mlsanity.com/paper/2210.16604/) (2022)
- [Detection of Homophobia & Transphobia in Dravidian Languages: Exploring Deep Learning Methods](https://mlsanity.com/paper/2304.01241/) (2023)
- [On the Challenges of Building Datasets for Hate Speech Detection](https://mlsanity.com/paper/2309.02912/) (2023)

## Abstract (arXiv)

Online social networks are ubiquitous and user-friendly. Nevertheless, it is vital to detect and moderate offensive content to maintain decency and empathy. However, mining social media texts is a complex task since users don't adhere to any fixed patterns. Comments can be written in any combination of languages and many of them may be low-resource. In this paper, we present our system for the LT-EDI shared task on detecting homophobia and transphobia in social media comments. We experiment with a number of monolingual and multilingual transformer based models such as mBERT along with a data augmentation technique for tackling class imbalance. Such pretrained large models have recently shown tremendous success on a variety of benchmark tasks in natural language processing. We observe their performance on a carefully annotated, real life dataset of YouTube comments in English as well as Tamil. Our submission achieved ranks 9, 6 and 3 with a macro-averaged F1-score of 0.42, 0.64 and 0.58 in the English, Tamil and Tamil-English subtasks respectively. The code for the system has been open sourced.