# Persona-aware Generative Model for Code-mixed Language

arXiv: https://arxiv.org/abs/2309.02915 · HTML: https://mlsanity.com/paper/2309.02915/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MuRIL: Multilingual Representations for Indian Languages](https://mlsanity.com/paper/2103.10730/) (2021)
- [A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data](https://mlsanity.com/paper/1911.04700/) (2019)
- [BERT Learns to Teach: Knowledge Distillation with Meta Learning](https://mlsanity.com/paper/2106.04570/) (2021)
- [Adversarial Feature Matching for Text Generation](https://mlsanity.com/paper/1706.03850/) (2017)
- [Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences](https://mlsanity.com/paper/1909.08582/) (2019)
- [Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://mlsanity.com/paper/2303.13592/) (2023)
- [Self-Attentive Associative Memory](https://mlsanity.com/paper/2002.03519/) (2020)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- ["My Way of Telling a Story": Persona based Grounded Story Generation](https://mlsanity.com/paper/1906.06401/) (2019)
- [Code-switched Language Models Using Dual RNNs and Same-Source Pretraining](https://mlsanity.com/paper/1809.01962/) (2018)
- [Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing](https://mlsanity.com/paper/2105.08807/) (2021)
- [A Deep Generative Model for Code-Switched Text](https://mlsanity.com/paper/1906.08972/) (2019)
- [Code-switching patterns can be an effective route to improve performance of downstream NLP applications: A case study of humour, sarcasm and hate speech detection](https://mlsanity.com/paper/2005.02295/) (2020)
- [HIT: A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representation](https://mlsanity.com/paper/2105.14600/) (2021)
- [A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation](https://mlsanity.com/paper/2110.06823/) (2021)
- [A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformer](https://mlsanity.com/paper/2204.12753/) (2022)

## Built on by (most cited first, 3 of 3)

- [Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency](https://mlsanity.com/paper/2512.00461/) (2025)
- [A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2501.13419/) (2025)
- [Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs](https://mlsanity.com/paper/2602.11181/) (2026)

## Abstract (arXiv)

Code-mixing and script-mixing are prevalent across online social networks and multilingual societies. However, a user's preference toward code-mixing depends on the socioeconomic status, demographics of the user, and the local context, which existing generative models mostly ignore while generating code-mixed texts. In this work, we make a pioneering attempt to develop a persona-aware generative model to generate texts resembling real-life code-mixed texts of individuals. We propose a Persona-aware Generative Model for Code-mixed Generation, PARADOX, a novel Transformer-based encoder-decoder model that encodes an utterance conditioned on a user's persona and generates code-mixed texts without monolingual reference data. We propose an alignment module that re-calibrates the generated sequence to resemble real-life code-mixed texts. PARADOX generates code-mixed texts that are semantically more meaningful and linguistically more valid. To evaluate the personification capabilities of PARADOX, we propose four new metrics -- CM BLEU, CM Rouge-1, CM Rouge-L and CM KS. On average, PARADOX achieves 1.6 points better CM BLEU, 47% better perplexity and 32% better semantic coherence than the non-persona-based counterparts.