mlsanityData to 7 Oct 2026

PrahokBART: A Pre-trained Sequence-to-Sequence Model for Khmer Natural Language Generation

arXiv 2512.13552 ↗First posted 15 Dec 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

This work introduces it PrahokBART, a compact pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. We focus on improving the pre-training corpus quality and addressing the linguistic issues of Khmer, which are ignored in existing multilingual models, by incorporating linguistic components such as word segmentation and normalization. We evaluate PrahokBART on three generative tasks: machine translation, text summarization, and headline generation, where our results demonstrate that it outperforms mBART50, a strong multilingual pre-trained model. Additionally, our analysis provides insights into the impact of each linguistic module and evaluates how effectively our model handles space during text generation, which is crucial for the naturalness of texts in Khmer.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 59 references that are in the map. First posted on arXiv 15 Dec 2025.

Builds on

38 of 59 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  10. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  12. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
26 more
  1. 2019How multilingual is Multilingual BERT?cited by 642
  2. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  3. 2019CamemBERT: a Tasty French Language Modelcited by 368
  4. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  5. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  6. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  7. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  8. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  9. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  10. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  11. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  12. 2020BARThez: a Skilled Pretrained French Sequence-to-Sequence Modelcited by 39
  13. 2020Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translationcited by 38
  14. 2021BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamesecited by 33
  15. 2024RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanizationcited by 29
  16. 2023Efficiently Adapting Pretrained Language Models To New Languagescited by 28
  17. 2022IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languagescited by 27
  18. 2022Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languagescited by 25
  19. 2021AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languagescited by 24
  20. 2022IT5: Text-to-text Pretraining for Italian Language Understanding and Generationcited by 20
  21. 2022How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?cited by 20
  22. 2022Improving Tokenisation by Alternative Treatment of Spacescited by 19
  23. 2024Cheetah: Natural Language Generation for 517 African Languagescited by 15
  24. 2021YANMTT: Yet Another Neural Machine Translation Toolkitcited by 7
  25. 2022LR-Sum: Summarization for Less-Resourced Languagescited by 5
  26. 2021Joint Khmer Word Segmentation and Part-of-Speech Tagging Using Deep Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026KSE-Web: An Analysis of Hybrid Retrieval and LLM-Assisted Query Expansion for Low-Resource Khmer Semantic Searchcited by 0