# KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media

arXiv: https://arxiv.org/abs/2007.13184 · HTML: https://mlsanity.com/paper/2007.13184/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2020.

## Built on by (most cited first, 30 of 84)

- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://mlsanity.com/paper/2103.06678/) (2021)
- [Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models](https://mlsanity.com/paper/2308.16149/) (2023)
- [ORCA: A Challenging Benchmark for Arabic Language Understanding](https://mlsanity.com/paper/2212.10758/) (2022)
- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [SpanEmo: Casting Multi-label Emotion Classification as Span-prediction](https://mlsanity.com/paper/2101.10038/) (2021)
- [Czert -- Czech BERT-like Model for Language Representation](https://mlsanity.com/paper/2103.13031/) (2021)
- [ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders](https://mlsanity.com/paper/2105.01279/) (2021)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text](https://mlsanity.com/paper/2311.03179/) (2023)
- [Large Language Models for Propaganda Span Annotation](https://mlsanity.com/paper/2311.09812/) (2023)
- [Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data](https://mlsanity.com/paper/2105.15071/) (2021)
- [SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 13 Languages](https://mlsanity.com/paper/2402.08638/) (2024)
- [Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection](https://mlsanity.com/paper/2109.12986/) (2021)
- [Leveraging Label Correlations in a Multi-label Setting: A Case Study in Emotion](https://mlsanity.com/paper/2210.15842/) (2022)
- [Mapping the Multilingual Margins: Intersectional Biases of Sentiment Analysis Systems in English, Spanish, and Arabic](https://mlsanity.com/paper/2204.03558/) (2022)
- [A Survey of Large Language Models for Arabic Language and its Dialects](https://mlsanity.com/paper/2410.20238/) (2024)
- [ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content](https://mlsanity.com/paper/2407.04247/) (2024)
- [LU-BZU at SemEval-2021 Task 2: Word2Vec and Lemma2Vec performance in Arabic Word-in-Context disambiguation](https://mlsanity.com/paper/2104.08110/) (2021)
- [Vietnamese Hate and Offensive Detection using PhoBERT-CNN and Social Media Streaming Data](https://mlsanity.com/paper/2206.00524/) (2022)
- [An Effective Deployment of Contrastive Learning in Multi-label Text Classification](https://mlsanity.com/paper/2212.00552/) (2022)
- [A Transformer-based Approach for Arabic Offline Handwritten Text Recognition](https://mlsanity.com/paper/2307.15045/) (2023)
- [AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking](https://mlsanity.com/paper/2104.13559/) (2021)
- [Corpus Considerations for Annotator Modeling and Scaling](https://mlsanity.com/paper/2404.02340/) (2024)
- [Are the Multilingual Models Better? Improving Czech Sentiment with Transformers](https://mlsanity.com/paper/2108.10640/) (2021)
- [AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing](https://mlsanity.com/paper/2306.06800/) (2023)
- [SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corpora](https://mlsanity.com/paper/2405.06239/) (2024)
- [Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentation](https://mlsanity.com/paper/2107.05684/) (2021)

## Abstract (arXiv)

In this paper, we describe our approach to utilize pre-trained BERT models with Convolutional Neural Networks for sub-task A of the Multilingual Offensive Language Identification shared task (OffensEval 2020), which is a part of the SemEval 2020. We show that combining CNN with BERT is better than using BERT on its own, and we emphasize the importance of utilizing pre-trained language models for downstream tasks. Our system, ranked 4th with macro averaged F1-Score of 0.897 in Arabic, 4th with score of 0.843 in Greek, and 3rd with score of 0.814 in Turkish. Additionally, we present ArabicBERT, a set of pre-trained transformer language models for Arabic that we share with the community.