# TIM: Teaching Large Language Models to Translate with Comparison

arXiv: https://arxiv.org/abs/2307.04408 · HTML: https://mlsanity.com/paper/2307.04408/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 10 Jul 2023.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [In-context Examples Selection for Machine Translation](https://mlsanity.com/paper/2212.02437/) (2022)
- [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://mlsanity.com/paper/2304.04675/) (2023)
- [Detecting Hallucinated Content in Conditional Neural Sequence Generation](https://mlsanity.com/paper/2011.02593/) (2020)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models](https://mlsanity.com/paper/2306.10968/) (2023)
- [ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback](https://mlsanity.com/paper/2304.02426/) (2023)
- [UniTE: Unified Translation Evaluation](https://mlsanity.com/paper/2204.13346/) (2022)
- [Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/2109.06604/) (2021)
- [Bridging Neural Machine Translation and Bilingual Dictionaries](https://mlsanity.com/paper/1610.07272/) (2016)
- [Chain-of-Dictionary Prompting Elicits Translation in Large Language Models](https://mlsanity.com/paper/2305.06575/) (2023)

## Built on by (most cited first, 30 of 50)

- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models](https://mlsanity.com/paper/2309.11674/) (2023)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement](https://mlsanity.com/paper/2402.16379/) (2024)
- [X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale](https://mlsanity.com/paper/2410.03115/) (2024)
- [Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrations](https://mlsanity.com/paper/2308.14186/) (2023)
- [SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection](https://mlsanity.com/paper/2402.16705/) (2024)
- [G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation](https://mlsanity.com/paper/2405.12915/) (2024)
- [Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning](https://mlsanity.com/paper/2310.03400/) (2023)
- [Improving Translation Faithfulness of Large Language Models via Augmenting Instructions](https://mlsanity.com/paper/2308.12674/) (2023)
- [Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2311.08380/) (2023)
- [TasTe: Teaching Large Language Models to Translate through Self-Reflection](https://mlsanity.com/paper/2406.08434/) (2024)
- [DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory](https://mlsanity.com/paper/2410.08143/) (2024)
- [Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models](https://mlsanity.com/paper/2410.11718/) (2024)
- [The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities](https://mlsanity.com/paper/2405.20089/) (2024)
- [Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?](https://mlsanity.com/paper/2404.14122/) (2024)
- [Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification](https://mlsanity.com/paper/2406.17534/) (2024)
- [Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Level](https://mlsanity.com/paper/2406.15741/) (2024)
- [Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning](https://mlsanity.com/paper/2410.02631/) (2024)
- [Lost in Literalism: How Supervised Training Shapes Translationese in LLMs](https://mlsanity.com/paper/2503.04369/) (2025)
- [Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding](https://mlsanity.com/paper/2311.02851/) (2023)
- [Simultaneous Machine Translation with Large Language Models](https://mlsanity.com/paper/2309.06706/) (2023)
- [A Preference-driven Paradigm for Enhanced Translation with Large Language Models](https://mlsanity.com/paper/2404.11288/) (2024)
- [Language Generation with Strictly Proper Scoring Rules](https://mlsanity.com/paper/2405.18906/) (2024)
- [AnyTrans: Translate AnyText in the Image with Large Scale Models](https://mlsanity.com/paper/2406.11432/) (2024)
- [Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation](https://mlsanity.com/paper/2410.07779/) (2024)
- [Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models](https://mlsanity.com/paper/2506.13044/) (2025)
- [Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attacks](https://mlsanity.com/paper/2408.08924/) (2024)
- [On-the-Fly Fusion of Large Language Models and Machine Translation](https://mlsanity.com/paper/2311.08306/) (2023)
- [Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Models](https://mlsanity.com/paper/2401.05861/) (2024)

## Abstract (arXiv)

Open-sourced large language models (LLMs) have demonstrated remarkable efficacy in various tasks with instruction tuning. However, these models can sometimes struggle with tasks that require more specialized knowledge such as translation. One possible reason for such deficiency is that instruction tuning aims to generate fluent and coherent text that continues from a given instruction without being constrained by any task-specific requirements. Moreover, it can be more challenging for tuning smaller LLMs with lower-quality training data. To address this issue, we propose a novel framework using examples in comparison to teach LLMs to learn translation. Our approach involves presenting the model with examples of correct and incorrect translations and using a preference loss to guide the model's learning. We evaluate our method on WMT2022 test sets and show that it outperforms existing methods. Our findings offer a new perspective on fine-tuning LLMs for translation tasks and provide a promising solution for generating high-quality translations. Please refer to Github for more details: https://github.com/lemon0830/TIM.