mlsanityData to 7 Oct 2026

Task-Specific Pre-Training and Cross Lingual Transfer for Code-Switched Data

arXiv 2102.12407 ↗First posted 24 Feb 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Using task-specific pre-training and leveraging cross-lingual transfer are two of the most popular ways to handle code-switched data. In this paper, we aim to compare the effects of both for the task of sentiment analysis. We work with two Dravidian Code-Switched languages - Tamil-Engish and Malayalam-English and four different BERT based models. We compare the effects of task-specific pre-training and cross-lingual transfer and find that task-specific pre-training results in superior zero-shot and supervised performance when compared to performance achieved by leveraging cross-lingual transfer from multilingual BERT models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 10 of its 23 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

10 of 23 references.

References · 10

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  4. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  5. 2019A Survey of Code-switched Speech and Language Processingcited by 73
  6. 2020Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Textcited by 67
  7. 2020SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweetscited by 62
  8. 2020A Sentiment Analysis Dataset for Code-Mixed Malayalam-Englishcited by 50
  9. 2018Sentiment Analysis of Code-Mixed Indian Languages: An Overview of SAIL_Code-Mixed Shared Task @ICON-2017cited by 23
  10. 2021SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identificationcited by 11

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 1; 2023: 0; 2024: 1.

5 papers.

  1. 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
  2. 2021Cross-lingual Transfer for Speech Processing using Acoustic Language Similaritycited by 8
  3. 2021Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Textcited by 7
  4. 2024Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexiconcited by 7
  5. 2021The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understandingcited by 4