# Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation

arXiv: https://arxiv.org/abs/2505.24683 · HTML: https://mlsanity.com/paper/2505.24683/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 77 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 30 of 37)

- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020)
- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary](https://mlsanity.com/paper/2010.00490/) (2020)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations](https://mlsanity.com/paper/2302.02187/) (2023)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [An Evaluation of the Human-Interpretability of Explanation](https://mlsanity.com/paper/1902.00006/) (2019)
- [TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks](https://mlsanity.com/paper/2310.00752/) (2023)
- [TICO-19: the Translation Initiative for Covid-19](https://mlsanity.com/paper/2007.01788/) (2020)
- [Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies](https://mlsanity.com/paper/2502.08554/) (2025)
- [Evaluating and Characterizing Human Rationales](https://mlsanity.com/paper/2010.04736/) (2020)
- [The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems](https://mlsanity.com/paper/2008.09100/) (2020)
- [Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2204.06916/) (2022)
- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [Translation Error Detection as Rationale Extraction](https://mlsanity.com/paper/2108.12197/) (2021)
- [Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors](https://mlsanity.com/paper/2310.16924/) (2023)
- [To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models](https://mlsanity.com/paper/2412.15584/) (2024)
- [Towards Explainable Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2306.13041/) (2023)
- [Unmet Needs and Opportunities for Mobile Translation AI](https://mlsanity.com/paper/2002.12387/) (2020)
- [Backtranslation Feedback Improves User Confidence in MT, Not Quality](https://mlsanity.com/paper/2104.05688/) (2021)
- [Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answering](https://mlsanity.com/paper/2504.07583/) (2025)
- [Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differences](https://mlsanity.com/paper/2312.01582/) (2023)

## Built on by (most cited first, 5 of 5)

- [An Interdisciplinary Approach to Human-Centered Machine Translation](https://mlsanity.com/paper/2506.13468/) (2025)
- [Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations](https://mlsanity.com/paper/2510.09994/) (2025)
- [Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms](https://mlsanity.com/paper/2604.23842/) (2026)
- [Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios](https://mlsanity.com/paper/2606.06177/) (2026)
- [Measuring User's Mental Models of Speech Translation in Human-AI Collaboration](https://mlsanity.com/paper/2606.24644/) (2026)

## Abstract (arXiv)

As people increasingly use AI systems in work and daily life, feedback mechanisms that help them use AI responsibly are urgently needed, particularly in settings where users are not equipped to assess the quality of AI predictions. We study a realistic Machine Translation (MT) scenario where monolingual users decide whether to share an MT output, first without and then with quality feedback. We compare four types of quality feedback: explicit feedback that directly give users an assessment of translation quality using (1) error highlights and (2) LLM explanations, and implicit feedback that helps users compare MT inputs and outputs through (3) backtranslation and (4) question-answer (QA) tables. We find that all feedback types, except error highlights, significantly improve both decision accuracy and appropriate reliance. Notably, implicit feedback, especially QA tables, yields significantly greater gains than explicit feedback in terms of decision accuracy, appropriate reliance, and user perceptions, receiving the highest ratings for helpfulness and trust, and the lowest for mental burden.