mlsanityData to 14 Oct 2026

PhoGPT: Generative Pre-training for Vietnamese

arXiv 2311.02945 ↗First posted 6 Nov 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

We open-source a state-of-the-art 4B-parameter generative model series for Vietnamese, which includes the base pre-trained monolingual model PhoGPT-4B and its chat variant, PhoGPT-4B-Chat. The base model, PhoGPT-4B, with exactly 3.7B parameters, is pre-trained from scratch on a Vietnamese corpus of 102B tokens, with an 8192 context length, employing a vocabulary of 20480 token types. The chat variant, PhoGPT-4B-Chat, is the modeling output obtained by fine-tuning PhoGPT-4B on a dataset of 70K instructional prompts and their responses, along with an additional 290K conversations. In addition, we also demonstrate its superior performance compared to previous open-source models. Our PhoGPT models are available at: https://github.com/VinAIResearch/PhoGPT

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 16 of its 21 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

16 of 21 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  11. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 149
  12. 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
4 more
  1. 2023LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Modelscited by 101
  2. 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 81
  3. 2021BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamesecited by 34
  4. 2023Efficient Finetuning Large Language Models For Vietnamese Chatbotcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 2.

8 papers.

  1. 2023VinaLLaMA: LLaMA-based Vietnamese Foundation Modelcited by 10
  2. 2024Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Modelscited by 6
  3. 2025GreenMind: A Next-Generation Vietnamese Large Language Model for Structured and Logical Reasoningcited by 3
  4. 2023ComOM at VLSP 2023: A Dual-Stage Framework with BERTology and Unified Multi-Task Instruction Tuning Model for Vietnamese Comparative Opinion Miningcited by 1
  5. 2024ViLLM-Eval: A Comprehensive Evaluation Suite for Vietnamese Large Language Modelscited by 1
  6. 2025VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Textcited by 1
  7. 2026ConnectED: A Curriculum-Aligned AI System for Vietnamese Instructional Lesson Planning and Student Learningcited by 0
  8. 2026VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Textcited by 0