Qwen-MT-Lumen at WMT2026: Instruction-Specialized Translation via Quality-Aware Training and On-Policy Distillation
On the map: Knowledge distillation → Knowledge distillation for LLMs
Abstract (arXiv)
We present Lumen, The open-source model is available at, our submission to the WMT 2026 General Machine Translation Shared Task. We focus on instruction-specialized translation, where systems must produce accurate translations while satisfying fine-grained requirements on style, terminology, and structured output. Lumen follows a four-stage pipeline. First, continued pre-training and supervised fine-tuning jointly strengthen multilingual translation competence and the ability to follow translation-specific instructions, such as terminology, style, and format constraints. Second, we extend the originally offline M^{2}PO preference-optimization framework into an online preference-optimization stage: the current policy samples candidate translations, an LLM judge assigns continuous direct-assessment scores, and the resulting multi-pair preferences are used to update the model. Third, on-policy distillation transfers stronger translation-instruction-following behavior from a larger teacher model to the Qwen3-14B student, improving translation-instruction adherence while retaining a 14B primary translation backbone. Finally, structure-aware inference and postprocessing preserve required formats, while selective repair addresses clear structural or decoding failures before submission. We participate in 23 translation directions. Under an absolute 1--5 LLM-judge rubric that evaluates translation-instruction following and translation quality separately, Lumen reaches an overall mean score of 4.60. Despite using a 14B translation backbone, it achieves performance comparable to leading proprietary systems in both translation quality and translation-instruction following.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 10 of its 18 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
10 of 18 references.
References · 10
- 2017Attention Is All You Needcited by 60,361
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2016Sequence-Level Knowledge Distillationcited by 801
- 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
- 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
- 2025M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs
- 2026Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning
- 2026IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
- 2026CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning
- 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
- 2025Improving Low-Resource Translation with Dictionary-Guided Fine-Tuning and RL: A Spanish-to-Wayuunaiki Study
- 2024X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructions
- 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters