mlsanityData to 7 Oct 2026

Improving Code Translation with Syntax-Guided and Semantic-aware Preference Optimization

arXiv 2605.13229 ↗First posted 13 May 2026cs.AI cs.SEcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

LLMs have shown immense potential for code translation, yet they often struggle to ensure both syntactic correctness and semantic consistency. While preference-based learning offers a promising alignment strategy, it is hindered by unreliable semantic rewards derived from sparse test cases or restrictive reference translations. We argue that a robust semantic reward for code translation must be derived directly from the source code. In this paper, we propose CTO to improve code translation with syntax-guided and semantic-aware preference optimization. Through contrastive learning, we train a cross-lingual semantic model to directly assess functional equivalence between source and translated code. By formulating code translation as a multi-objective optimization problem, this robust semantic signal is seamlessly unified with compiler-based syntactic feedback within the direct preference optimization framework. Extensive experiments on C++, Java, and Python translations demonstrate that CTO significantly outperforms existing baselines and alternative preference optimization strategies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 36 of its 41 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

36 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Dynamic Scaling of Unit Tests for Code Reward Modeling2 Jan

Other references · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  10. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  11. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  12. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
23 more
  1. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  2. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  3. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  4. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  5. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  6. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  7. 2020Unsupervised Translation of Programming Languagescited by 262
  8. 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
  9. 2023CodeBERTScore: Evaluating Code Generation with Pretrained Models of Codecited by 101
  10. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  11. 2021Leveraging Automated Unit Tests for Unsupervised Code Translationcited by 82
  12. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  13. 2024Exploring and Unleashing the Power of Large Language Models in Automated Code Translationcited by 57
  14. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 55
  15. 2022XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligencecited by 54
  16. 2023xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrievalcited by 46
  17. 2022Code Translation with Compiler Representationscited by 42
  18. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
  19. 2024CodeDPO: Aligning Code Models with Self Generated and Verified Source Codecited by 32
  20. 2024Code-Optimise: Self-Generated Preference Data for Correctness and Efficiencycited by 16
  21. 2023Program Translation via Code Distillationcited by 10
  22. 2023Syntax and Domain Aware Model for Unsupervised Program Translationcited by 7
  23. 2024A Joint Learning Model with Variational Interaction for Multilingual Program Translationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Code Preference via Synthetic Evolution
  2. 2025Learning to Align Human Code Preferences
  3. 2025Enhancing the Non-Functional Quality Compliance of LLM-Generated Code through Quality-Aware Preference Learning
  4. 2026Bootstrapping Niche Multilingual Code Translation via Reinforcement Learning with Execution-Based Verifiable Supervision
  5. 2026Enhancing LLM-Based Code Translation with Verified Multi-Semantic Representations ↗
  6. 2025On-Policy Optimization with Group Equivalent Preference for Multi-Programming Language Understanding
  7. 2026Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation
  8. 2026Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus