Gradable ChatGPT Translation Evaluation
Not on the map. Why?
Abstract (arXiv)
ChatGPT, as a language model based on large-scale pre-training, has exerted a profound influence on the domain of machine translation. In ChatGPT, a "Prompt" refers to a segment of text or instruction employed to steer the model towards generating a specific category of response. The design of the translation prompt emerges as a key aspect that can wield influence over factors such as the style, precision and accuracy of the translation to a certain extent. However, there is a lack of a common standard and methodology on how to design and select a translation prompt. Accordingly, this paper proposes a generic taxonomy, which defines gradable translation prompts in terms of expression type, translation style, POS information and explicit statement, thus facilitating the construction of prompts endowed with distinct attributes tailored for various translation tasks. Specific experiments and cases are selected to validate and illustrate the effectiveness of the method.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 18 Jan 2024.
Builds on
26 of 44 references.
References · 26
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
- 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
- 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
- 2023Language Models can Solve Computer Taskscited by 372
14 more
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
- 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
- 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 176
- 2023Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Modelscited by 87
- 2022Legal Prompt Engineering for Multilingual Legal Judgement Predictioncited by 55
- 2020A Survey of Deep Learning Techniques for Neural Machine Translationcited by 50
- 2023TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Taskscited by 49
- 2020Incorporating BERT into Parallel Sequence Decoding with Adapterscited by 39
- 2021MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translatorscited by 35
- 2023A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understandingcited by 31
- 2022Toward Human-Like Evaluation for Natural Language Generation with Error Analysiscited by 18
- 2022Tencent AI Lab - Shanghai Jiao Tong University Low-Resource Translation System for the WMT22 Translation Taskcited by 8
- 2019Context in Neural Machine Translation: A Review of Models and Evaluationscited by 7
- 2019Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoringcited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.