mlsanityData to 14 Oct 2026

Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation

arXiv 2505.24683 ↗First posted 30 May 2025cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

As people increasingly use AI systems in work and daily life, feedback mechanisms that help them use AI responsibly are urgently needed, particularly in settings where users are not equipped to assess the quality of AI predictions. We study a realistic Machine Translation (MT) scenario where monolingual users decide whether to share an MT output, first without and then with quality feedback. We compare four types of quality feedback: explicit feedback that directly give users an assessment of translation quality using (1) error highlights and (2) LLM explanations, and implicit feedback that helps users compare MT inputs and outputs through (3) backtranslation and (4) question-answer (QA) tables. We find that all feedback types, except error highlights, significantly improve both decision accuracy and appropriate reliance. Notably, implicit feedback, especially QA tables, yields significantly greater gains than explicit feedback in terms of decision accuracy, appropriate reliance, and user perceptions, receiving the highest ratings for helpfulness and trust, and the lowest for mental burden.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 37 of its 77 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

37 of 77 references.

References · 37

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  2. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  3. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  4. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  5. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  6. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  7. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 253
  8. 2019Explainable Machine Learning in Deploymentcited by 211
  9. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 167
  10. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  11. 2020Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summarycited by 88
  12. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 86
25 more
  1. 2023Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanationscited by 83
  2. 2020Human Factors in Model Interpretability: Industry Practices, Challenges, and Needscited by 74
  3. 2019An Evaluation of the Human-Interpretability of Explanationcited by 71
  4. 2023TIGERScore: Towards Building Explainable Metric for All Text Generation Taskscited by 70
  5. 2020TICO-19: the Translation Initiative for Covid-19cited by 68
  6. 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistenciescited by 39
  7. 2020Evaluating and Characterizing Human Rationalescited by 36
  8. 2020The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systemscited by 34
  9. 2022Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Makingcited by 28
  10. 2022Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Makingcited by 25
  11. 2021Translation Error Detection as Rationale Extractioncited by 22
  12. 2023Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errorscited by 22
  13. 2024To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Modelscited by 18
  14. 2023Towards Explainable Evaluation Metrics for Machine Translationcited by 15
  15. 2020Unmet Needs and Opportunities for Mobile Translation AIcited by 14
  16. 2021Backtranslation Feedback Improves User Confidence in MT, Not Qualitycited by 10
  17. 2025Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answeringcited by 10
  18. 2023Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differencescited by 8
  19. 2025Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactionscited by 8
  20. 2025AskQE: Question Answering as Automatic Evaluation for Machine Translationcited by 7
  21. 2025Explaining and Improving Information Complementarities in Multi-Agent Decision-makingcited by 6
  22. 2025Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistancecited by 6
  23. 2024Beyond Recommender: An Exploratory Study of the Effects of Different AI Roles in AI-Assisted Decision Makingcited by 5
  24. 2025Sustaining Human Agency, Attending to Its Cost: An Investigation into Generative AI Design for Non-Native Speakers' Language Usecited by 5
  25. 2019Outbound Translation User Interface Ptakopet: A Pilot Studycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025An Interdisciplinary Approach to Human-Centered Machine Translationcited by 6
  2. 2025Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translationscited by 1
  3. 2026Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologismscited by 1
  4. 2026Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarioscited by 0
  5. 2026Measuring User's Mental Models of Speech Translation in Human-AI Collaborationcited by 0