# CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts

arXiv: https://arxiv.org/abs/2404.12618 · HTML: https://mlsanity.com/paper/2404.12618/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 19 Apr 2024.

## Builds on (most cited first, 27 of 27)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models](https://mlsanity.com/paper/2010.12858/) (2020)
- [CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP](https://mlsanity.com/paper/2006.06402/) (2020)
- [ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://mlsanity.com/paper/2106.16038/) (2021)
- [Consistency Regularization for Cross-Lingual Fine-Tuning](https://mlsanity.com/paper/2106.08226/) (2021)
- [Dense Hierarchical Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2110.15439/) (2021)
- [Enhancing Cross-lingual Transfer by Manifold Mixup](https://mlsanity.com/paper/2205.04182/) (2022)
- [FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding](https://mlsanity.com/paper/2009.05166/) (2020)
- [Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability](https://mlsanity.com/paper/2203.10753/) (2022)
- [CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection](https://mlsanity.com/paper/2004.01881/) (2020)
- [CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks](https://mlsanity.com/paper/2310.14623/) (2023)
- [Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection](https://mlsanity.com/paper/2010.02481/) (2020)
- [Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction](https://mlsanity.com/paper/2110.15430/) (2021)
- [On the Role of Parallel Data in Cross-lingual Transfer Learning](https://mlsanity.com/paper/2212.10173/) (2022)
- [Enhancing Cross-lingual Transfer via Phonemic Transcription Integration](https://mlsanity.com/paper/2307.04361/) (2023)
- [Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation](https://mlsanity.com/paper/2302.11131/) (2023)
- [Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning](https://mlsanity.com/paper/2308.04712/) (2023)

## Built on by (most cited first, 4 of 4)

- [Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languages](https://mlsanity.com/paper/2411.02398/) (2024)
- [Not Truly Multilingual: Script Consistency as a Missing Dimension in VLM Evaluation](https://mlsanity.com/paper/2606.17188/) (2026)
- [Beyond 'One Language, One Script': Quantifying Orthographic Bias in Multilingual VLMs with PuMVR](https://mlsanity.com/paper/2606.20770/) (2026)
- [CoSE-E: A Benchmark for Code-switched Speech Evaluation in Enterprise Settings](https://mlsanity.com/paper/2609.35645/) (2026)

## Abstract (arXiv)

Naively assuming English as a source language may hinder cross-lingual transfer for many languages by failing to consider the importance of language contact. Some languages are more well-connected than others, and target languages can benefit from transferring from closely related languages; for many languages, the set of closely related languages does not include English. In this work, we study the impact of source language for cross-lingual transfer, demonstrating the importance of selecting source languages that have high contact with the target language. We also construct a novel benchmark dataset for close contact Chinese-Japanese-Korean-Vietnamese (CJKV) languages to further encourage in-depth studies of language contact. To comprehensively capture contact between these languages, we propose to integrate Romanized transcription beyond textual scripts via Contrastive Learning objectives, leading to enhanced cross-lingual representations and effective zero-shot cross-lingual transfer.