# Comparative Study of Pre-Trained BERT and Large Language Models for Code-Mixed Named Entity Recognition

arXiv: https://arxiv.org/abs/2509.02514 · HTML: https://mlsanity.com/paper/2509.02514/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [Multilingual Offensive Language Identification with Cross-lingual Embeddings](https://mlsanity.com/paper/2010.05324/) (2020)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Models](https://mlsanity.com/paper/2204.08398/) (2022)
- [Learning Fast Adaptation on Cross-Accented Speech Recognition](https://mlsanity.com/paper/2003.01901/) (2020)
- [Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task](https://mlsanity.com/paper/1906.04138/) (2019)
- [A hybrid deep-learning approach for complex biochemical named entity recognition](https://mlsanity.com/paper/2012.10824/) (2020)
- [CMNEROne at SemEval-2022 Task 11: Code-Mixed Named Entity Recognition by leveraging multilingual data](https://mlsanity.com/paper/2206.07318/) (2022)
- [Comparative Study of Pre-Trained BERT Models for Code-Mixed Hindi-English Data](https://mlsanity.com/paper/2305.15722/) (2023)
- [My Boli: Code-mixed Marathi-English Corpora, Pretrained Language Models and Evaluation Benchmarks](https://mlsanity.com/paper/2306.14030/) (2023)

## Abstract (arXiv)

Named Entity Recognition (NER) in code-mixed text, particularly Hindi-English (Hinglish), presents unique challenges due to informal structure, transliteration, and frequent language switching. This study conducts a comparative evaluation of code-mixed fine-tuned models and non-code-mixed multilingual models, along with zero-shot generative large language models (LLMs). Specifically, we evaluate HingBERT, HingMBERT, and HingRoBERTa (trained on code-mixed data), and BERT Base Cased, IndicBERT, RoBERTa and MuRIL (trained on non-code-mixed multilingual data). We also assess the performance of Google Gemini in a zero-shot setting using a modified version of the dataset with NER tags removed. All models are tested on a benchmark Hinglish NER dataset using Precision, Recall, and F1-score. Results show that code-mixed models, particularly HingRoBERTa and HingBERT-based fine-tuned models, outperform others - including closed-source LLMs like Google Gemini - due to domain-specific pretraining. Non-code-mixed models perform reasonably but show limited adaptability. Notably, Google Gemini exhibits competitive zero-shot performance, underlining the generalization strength of modern LLMs. This study provides key insights into the effectiveness of specialized versus generalized models for code-mixed NER tasks.