# Share What You Already Know: Cross-Language-Script Transfer and Alignment for Sentiment Detection in Code-Mixed Data

arXiv: https://arxiv.org/abs/2402.04542 · HTML: https://mlsanity.com/paper/2402.04542/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Cross-Lingual Ability of Multilingual BERT: An Empirical Study](https://mlsanity.com/paper/1912.07840/) (2019)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Multilingual Alignment of Contextual Word Representations](https://mlsanity.com/paper/2002.03518/) (2020)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study](https://mlsanity.com/paper/2009.14304/) (2020)
- [Aksharantar: Open Indic-language Transliteration datasets and models for the Next Billion Users](https://mlsanity.com/paper/2205.03018/) (2022)
- [kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification](https://mlsanity.com/paper/2009.03673/) (2020)
- [NLP-CIC at SemEval-2020 Task 9: Analysing sentiment in code-switching language using a simple deep-learning classifier](https://mlsanity.com/paper/2009.03397/) (2020)
- [Linguistic Taboos and Euphemisms in Nepali](https://mlsanity.com/paper/2007.13798/) (2020)
- [Elevating Code-mixed Text Handling through Auditory Information of Words](https://mlsanity.com/paper/2310.18155/) (2023)

## Abstract (arXiv)

Code-switching entails mixing multiple languages. It is an increasingly occurring phenomenon in social media texts. Usually, code-mixed texts are written in a single script, even though the languages involved have different scripts. Pre-trained multilingual models primarily utilize the data in the native script of the language. In existing studies, the code-switched texts are utilized as they are. However, using the native script for each language can generate better representations of the text owing to the pre-trained knowledge. Therefore, a cross-language-script knowledge sharing architecture utilizing the cross attention and alignment of the representations of text in individual language scripts was proposed in this study. Experimental results on two different datasets containing Nepali-English and Hindi-English code-switched texts, demonstrate the effectiveness of the proposed method. The interpretation of the model using model explainability technique illustrates the sharing of language-specific knowledge between language-specific representations.