mlsanityData to 14 Oct 2026

Structured Document Translation via Format Reinforcement Learning

arXiv 2512.05100 ↗First posted 4 Dec 2025cs.CL cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Recent works on structured text translation remain limited to the sentence level, as they struggle to effectively handle the complex document-level XML or HTML structures. To address this, we propose Format Reinforcement Learning (FormatRL), which employs Group Relative Policy Optimization on top of a supervised fine-tuning model to directly optimize novel structure-aware rewards: 1) TreeSim, which measures structural similarity between predicted and reference XML trees and 2) Node-chrF, which measures translation quality at the level of XML nodes. Additionally, we apply StrucAUC, a fine-grained metric distinguishing between minor errors and major structural failures. Experiments on the SAP software-documentation benchmark demonstrate improvements across six metrics and an analysis further shows how different reward functions contribute to improvements in both structural and translation quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 36 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

16 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Learning to Generate Structured Output with Schema Reinforcement Learning26 Feb

Other references · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  8. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  9. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  10. 2024Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translationcited by 62
  11. 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
  12. 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
3 more
  1. 2020A High-Quality Multilingual Dataset for Structured Documentation Translationcited by 10
  2. 2024ReFF: Reinforcing Format Faithfulness in Language Models across Varied Taskscited by 9
  3. 2020A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
  2. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
  3. 2026CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation
  4. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
  5. 2025Align to Structure: Aligning Large Language Models with Structural Information
  6. 2025DocReward: A Document Reward Model for Structuring and Stylizing
  7. 2026Better Literary Translation: A Multi-Aspect Data Generation and LLM Training Approach
  8. 2026Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation