mlsanityData to 14 Oct 2026

Echoes Across Vietnam's Highlands, Delta, and Coast: A Multilingual Corpus for Cham, Khmer, and Tay-Nung

arXiv 2607.08362 ↗First posted 9 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Vietnam's ethnic minority languages are almost absent from the field of Natural Language Processing (NLP), and the challenge goes beyond data scarcity: Cham, Khmer, and Tay-Nung differ sharply in script, Vietnamese contact, and standardization, conditions under which standard multilingual adaptation can learn the wrong signals. We introduce CKTN, the first corpus and benchmark for these languages (44,367 documents, 24M subword tokens), spanning continued pretraining, category classification, and summary-document retrieval. We show that existing multilingual encoders severely fragment these languages, and that common adaptation metrics can mislead: models may lower language-modeling loss or excel at lexical-overlap retrieval while still failing at semantic generalization across documents. We address this with a script-aware adaptation recipe - vocabulary augmentation combined with calibrated replaced-token pretraining - that prevents the discriminator from exploiting trivial script mismatches. The result is an encoder with substantially less fragmentation and the strongest classification performance among evaluated models, exposing the limits of lexical-overlap retrieval as an evaluation signal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 56 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

17 of 56 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  6. 2019How multilingual is Multilingual BERT?cited by 645
  7. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 146
  8. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  9. 2020Rethinking embedding coupling in pre-trained language modelscited by 103
  10. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  11. 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
  12. 2023Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languagescited by 62
5 more
  1. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  2. 2020AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understandingcited by 43
  3. 2024Fast Vocabulary Transfer for Language Model Compressioncited by 42
  4. 2020Exploiting News Article Structure for Automatic Corpus Generation of Entailment Datasetscited by 10
  5. 2024Targeted Multilingual Adaptation for Low-resource Language Familiescited by 9

Built on by

Not cited yet.