# All that is English may be Hindi: Enhancing language identification through automatic ranking of likeliness of word borrowing in social media

arXiv: https://arxiv.org/abs/1707.08446 · HTML: https://mlsanity.com/paper/1707.08446/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2017.

## Built on by (most cited first, 7 of 7)

- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Representativeness as a Forgotten Lesson for Multilingual and Code-switched Data Collection and Preparation](https://mlsanity.com/paper/2310.20470/) (2023)
- [GupShup: An Annotated Corpus for Abstractive Summarization of Open-Domain Code-Switched Conversations](https://mlsanity.com/paper/2104.08578/) (2021)
- [Borrowing or Codeswitching? Annotating for Finer-Grained Distinctions in Language Mixing](https://mlsanity.com/paper/2206.04973/) (2022)
- [The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding](https://mlsanity.com/paper/2107.09931/) (2021)
- [Revealing the impact of synthetic native samples and multi-tasking strategies in Hindi-English code-mixed humour and sarcasm detection](https://mlsanity.com/paper/2412.12761/) (2024)
- [Are Language Models Borrowing-Blind? A Multilingual Evaluation of Loanword Identification across 10 Languages](https://mlsanity.com/paper/2510.26254/) (2025)

## Abstract (arXiv)

In this paper, we present a set of computational methods to identify the likeliness of a word being borrowed, based on the signals from social media. In terms of Spearman correlation coefficient values, our methods perform more than two times better (nearly 0.62) in predicting the borrowing likeliness compared to the best performing baseline (nearly 0.26) reported in literature. Based on this likeliness estimate we asked annotators to re-annotate the language tags of foreign words in predominantly native contexts. In 88 percent of cases the annotators felt that the foreign language tag should be replaced by native language tag, thus indicating a huge scope for improvement of automatic language identification systems.