Translation Analytics for Freelancers: I. Introduction, Data Preparation, Baseline Evaluations
Not on the map. Why?
Abstract (arXiv)
This is the first in a series of papers exploring the rapidly expanding new opportunities arising from recent progress in language technologies for individual translators and language service providers with modest resources. The advent of advanced neural machine translation systems, large language models, and their integration into workflows via computer-assisted translation tools and translation management systems have reshaped the translation landscape. These advancements enable not only translation but also quality evaluation, error spotting, glossary generation, and adaptation to domain-specific needs, creating new technical opportunities for freelancers. In this series, we aim to empower translators with actionable methods to harness these advancements. Our approach emphasizes Translation Analytics, a suite of evaluation techniques traditionally reserved for large-scale industry applications but now becoming increasingly available for smaller-scale users. This first paper introduces a practical framework for adapting automatic evaluation metrics -- such as BLEU, chrF, TER, and COMET -- to freelancers' needs. We illustrate the potential of these metrics using a trilingual corpus derived from a real-world project in the medical domain and provide statistical analysis correlating human evaluations with automatic scores. Our findings emphasize the importance of proactive engagement with emerging technologies to not only adapt but thrive in the evolving professional environment.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 54 references that are in the map. First posted on arXiv 20 Apr 2025.
Builds on
22 of 54 references.
References · 22
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
- 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
- 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
- 2023The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluationcited by 95
- 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
- 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
- 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
- 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 56
10 more
- 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
- 2024TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinementcited by 39
- 2024Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Textscited by 31
- 2024Guiding Large Language Models to Post-Edit Machine Translation with Error Annotationscited by 22
- 2024How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizescited by 17
- 2024Did Translation Models Get More Robust Without Anyone Even Noticing?cited by 9
- 2024Instruction-tuned Large Language Models for Machine Translation in the Medical Domaincited by 4
- 2020NLPStatTest: A Toolkit for Comparing NLP System Performancecited by 3
- 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
- 2024Prompting Large Language Models with Human Error Markings for Self-Correcting Machine Translationcited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.