mlsanityData to 7 Oct 2026

SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identification

arXiv 2102.01051 ↗First posted 1 Feb 2021cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper we present our submission for the EACL 2021-Shared Task on Offensive Language Identification in Dravidian languages. Our final system is an ensemble of mBERT and XLM-RoBERTa models which leverage task-adaptive pre-training of multilingual BERT models with a masked language modeling objective. Our system was ranked 1st for Kannada, 2nd for Malayalam and 3rd for Tamil.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

15 of 27 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  6. 2019How multilingual is Multilingual BERT?cited by 642
  7. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  8. 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
  9. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 101
  10. 2020LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluationcited by 92
  11. 2019A Survey of Code-switched Speech and Language Processingcited by 73
  12. 2020Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Textcited by 67
3 more
  1. 2020SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweetscited by 62
  2. 2020A Sentiment Analysis Dataset for Code-Mixed Malayalam-Englishcited by 50
  3. 2020DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionariescited by 20

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 3; 2023: 0; 2024: 0; 2025: 1.

11 papers.

  1. 2021CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixingcited by 8
  2. 2021Unsupervised Self-Training for Sentiment Analysis of Code-Switched Datacited by 7
  3. 2021Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labelingcited by 6
  4. 2021Task-Specific Pre-Training and Cross Lingual Transfer for Code-Switched Datacited by 5
  5. 2021Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Postscited by 4
  6. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  7. 2021Developing Successful Shared Tasks on Offensive Language Identification for Dravidian Languagescited by 1
  8. 2022AIR-JPMC@SMM4H'22: Classifying Self-Reported Intimate Partner Violence in Tweets with Multiple BERT-based Modelscited by 1
  9. 2022Automatic Identification of Motivation for Code-Switching in Speech Transcriptscited by 1
  10. 2021Multilingual Text Classification for Dravidian Languagescited by 0
  11. 2022Sentiment Classification of Code-Switched Text using Pre-trained Multilingual Embeddings and Segmentationcited by 0