# PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues

arXiv: https://arxiv.org/abs/2601.17277 · HTML: https://mlsanity.com/paper/2601.17277/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 70 references that are in the map.
- First posted on arXiv 24 Jan 2026.

## Builds on (most cited first, 29 of 29)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Neural Text Summarization: A Critical Evaluation](https://mlsanity.com/paper/1908.08960/) (2019)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation](https://mlsanity.com/paper/2005.04322/) (2020)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences](https://mlsanity.com/paper/1909.08582/) (2019)
- [Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://mlsanity.com/paper/2303.13592/) (2023)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation](https://mlsanity.com/paper/2112.06223/) (2021)
- [PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation](https://mlsanity.com/paper/2004.09447/) (2020)
- [L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Models](https://mlsanity.com/paper/2204.08398/) (2022)
- [From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text](https://mlsanity.com/paper/2107.06483/) (2021)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task](https://mlsanity.com/paper/1906.04138/) (2019)
- [From English to Code-Switching: Transfer Learning with Strong Morphological Clues](https://mlsanity.com/paper/1909.05158/) (2019)
- [Code-Switching Language Modeling using Syntax-Aware Multi-Task Learning](https://mlsanity.com/paper/1805.12070/) (2018)
- [CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarization](https://mlsanity.com/paper/2303.04092/) (2023)
- [MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluation](https://mlsanity.com/paper/2107.11534/) (2021)
- [IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languages](https://mlsanity.com/paper/2311.12405/) (2023)
- [hinglishNorm -- A Corpus of Hindi-English Code Mixed Sentences for Text Normalization](https://mlsanity.com/paper/2010.08974/) (2020)
- [DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching](https://mlsanity.com/paper/2509.17768/) (2025)
- [L3Cube-MahaSum: A Comprehensive Dataset and BART Models for Abstractive Text Summarization in Marathi](https://mlsanity.com/paper/2410.09184/) (2024)
- [Euska~nolDS: A Naturally Sourced Corpus for Basque-Spanish Code-Switching](https://mlsanity.com/paper/2502.03188/) (2025)

## Built on by (most cited first, 1 of 1)

- [Code-Switching Reveals Language Anchoring in Multilingual LLMs](https://mlsanity.com/paper/2606.19668/) (2026)

## Abstract (arXiv)

Code-switching is a widespread practice among the world's multilingual majority, yet few benchmarks accurately reflect its complexity in everyday communication. We present PingPong, a benchmark for natural multi-party code-switching dialogues covering five language-combination variations, some of which are trilingual. Our dataset consists of human-authored conversations among 2 to 4 participants covering authentic, multi-threaded structures where replies frequently reference much earlier points in the dialogue. We demonstrate that our data is significantly more natural and structurally diverse than machine-generated alternatives, offering greater variation in message length, speaker dominance, and reply distance. Based on these dialogues, we define three downstream tasks: Question Answering, Dialogue Summarization, and Topic Classification. Evaluations of several state-of-the-art language models on PingPong reveal that performance remains limited on code-switched inputs, underscoring the urgent need for more robust NLP systems capable of addressing the intricacies of real-world multilingual discourse.