# UTNLP at SemEval-2021 Task 5: A Comparative Analysis of Toxic Span Detection using Attention-based, Named Entity Recognition, and Ensemble Models

arXiv: https://arxiv.org/abs/2104.04770 · HTML: https://mlsanity.com/paper/2104.04770/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 10 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media](https://mlsanity.com/paper/1804.04257/) (2018)
- [All You Need is "Love": Evading Hate-speech Detection](https://mlsanity.com/paper/1808.09115/) (2018)
- [Analyzing the Targets of Hate in Online Social Media](https://mlsanity.com/paper/1603.07709/) (2016)
- [Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction](https://mlsanity.com/paper/2010.03260/) (2020)
- [Dissecting Span Identification Tasks with Performance Prediction](https://mlsanity.com/paper/2010.02587/) (2020)
- [Progress Notes Classification and Keyword Extraction using Attention-based Deep Learning Models with BERT](https://mlsanity.com/paper/1910.05786/) (2019)
- [HSD Shared Task in VLSP Campaign 2019:Hate Speech Detection for Social Good](https://mlsanity.com/paper/2007.06493/) (2020)
- [Towards non-toxic landscapes: Automatic toxic comment detection using DNN](https://mlsanity.com/paper/1911.08395/) (2019)
- [Aspect Based Sentiment Analysis with Aspect-Specific Opinion Spans](https://mlsanity.com/paper/2010.02696/) (2020)

## Abstract (arXiv)

Detecting which parts of a sentence contribute to that sentence's toxicity -- rather than providing a sentence-level verdict of hatefulness -- would increase the interpretability of models and allow human moderators to better understand the outputs of the system. This paper presents our team's, UTNLP, methodology and results in the SemEval-2021 shared task 5 on toxic spans detection. We test multiple models and contextual embeddings and report the best setting out of all. The experiments start with keyword-based models and are followed by attention-based, named entity-based, transformers-based, and ensemble models. Our best approach, an ensemble model, achieves an F1 of 0.684 in the competition's evaluation phase.