mlsanityData to 7 Oct 2026

Efficient Training of Robust Traditional Chinese LLaMA-1B on a Single Consumer GPU: Continual Pre-training, SFT, and DPO

arXiv 2510.01616 ↗First posted 2 Oct 2025cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Small Language Models (SLMs) enable cost-effective, on-device and latency-sensitive AI applications, yet their deployment in Traditional Chinese (TC) remains hindered by token-level instability - models unpredictably emit non-TC characters or code-switch into other languages. We address this practical reliability gap by creating PureTC-1B, a three-stage stabilization pipeline for Llama-3.2-1B-Instruct (an open-weight, instruction-tuned model released by Meta) using parameter-efficient LoRA adapters. Our method combines Continual Pre-Training (CPT) on TC-centric corpora, Supervised Fine-Tuning (SFT) with instruction data, and Direct Preference Optimization (DPO) using TC-adherence preferences to improve monolingual robustness without full-model retraining. On a benchmark designed to simulate real-world usage, PureTC-1B achieves a 51.3% relative reduction (micro-average) in non-TC output tokens versus the base model. On a Named Entity Translation (NET) task, PureTC-1B further reduces incorrect-language tokens by 77.2% relative to Llama-3B and 57.2% relative to Qwen-1.5B, indicating that robust TC adherence is attainable even at the 1B scale. The pipeline is reproducible, adapter-only, and hardware-friendly, offering practitioners a practical recipe to enhance language stability for TC and potentially other non-English languages.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 8 of its 20 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

8 of 20 references.

References · 8

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  4. 2023Taiwan LLM: Bridging the Linguistic Divide with a Culturally Aligned Language Modelcited by 29
  5. 2024Code-Switching Curriculum Learning for Multilingual Transfer in LLMscited by 26
  6. 2025Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Trainingcited by 15
  7. 2025Lost in the Mix: Evaluating LLM Understanding of Code-Switched Textcited by 9
  8. 2025Breaking Language Barriers: Equitable Performance in Multilingual Language Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Effective and Efficient Continual Pre-training of Large Language Models
  2. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
  3. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
  4. 2025Zebra-Llama: Towards Extremely Efficient Hybrid Models
  5. 2023An Empirical Study of Instruction-tuning Large Language Models in Chinese
  6. 2023Accelerating LLaMA Inference by Enabling Intermediate Layer Decoding via Instruction Tuning with LITE
  7. 2025Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis
  8. 2024CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback