# Automatic Sexism Detection with Multilingual Transformer Models

arXiv: https://arxiv.org/abs/2106.04908 · HTML: https://mlsanity.com/paper/2106.04908/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model](https://mlsanity.com/paper/2008.06460/) (2020)

## Built on by (most cited first, 9 of 9)

- [Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse](https://mlsanity.com/paper/2410.00175/) (2024)
- [Divided by discipline? A systematic literature review on the quantification of online sexism and misogyny using a semi-automated approach](https://mlsanity.com/paper/2409.20204/) (2024)
- [SSS at SemEval-2023 Task 10: Explainable Detection of Online Sexism using Majority Voted Fine-Tuned Transformers](https://mlsanity.com/paper/2304.03518/) (2023)
- [Stanford MLab at SemEval-2023 Task 10: Exploring GloVe- and Transformer-Based Methods for the Explainable Detection of Online Sexism](https://mlsanity.com/paper/2305.04356/) (2023)
- [IUST_NLP at SemEval-2023 Task 10: Explainable Detecting Sexism with Transformers and Task-adaptive Pretraining](https://mlsanity.com/paper/2305.06892/) (2023)
- [AI-UPV at EXIST 2023 -- Sexism Characterization Using Large Language Models Under The Learning with Disagreements Regime](https://mlsanity.com/paper/2307.03385/) (2023)
- [A Holistic Indicator of Polarization to Measure Online Sexism](https://mlsanity.com/paper/2404.02205/) (2024)
- [A Survey on Stereotype Detection in Natural Language Processing](https://mlsanity.com/paper/2505.17642/) (2025)
- [Dataset Creation and Baseline Models for Sexism Detection in Hausa](https://mlsanity.com/paper/2510.27038/) (2025)

## Abstract (arXiv)

Sexism has become an increasingly major problem on social networks during the last years. The first shared task on sEXism Identification in Social neTworks (EXIST) at IberLEF 2021 is an international competition in the field of Natural Language Processing (NLP) with the aim to automatically identify sexism in social media content by applying machine learning methods. Thereby sexism detection is formulated as a coarse (binary) classification problem and a fine-grained classification task that distinguishes multiple types of sexist content (e.g., dominance, stereotyping, and objectification). This paper presents the contribution of the AIT_FHSTP team at the EXIST2021 benchmark for both tasks. To solve the tasks we applied two multilingual transformer models, one based on multilingual BERT and one based on XLM-R. Our approach uses two different strategies to adapt the transformers to the detection of sexist content: first, unsupervised pre-training with additional data and second, supervised fine-tuning with additional and augmented data. For both tasks our best model is XLM-R with unsupervised pre-training on the EXIST data and additional datasets and fine-tuning on the provided dataset. The best run for the binary classification (task 1) achieves a macro F1-score of 0.7752 and scores 5th rank in the benchmark; for the multiclass classification (task 2) our best submission scores 6th rank with a macro F1-score of 0.5589.