# Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations

arXiv: https://arxiv.org/abs/2510.09994 · HTML: https://mlsanity.com/paper/2510.09994/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 11 Oct 2025.

## Builds on (most cited first, 20 of 20)

- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback](https://mlsanity.com/paper/2311.09336/) (2023)
- [The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems](https://mlsanity.com/paper/2008.09100/) (2020)
- [The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results](https://mlsanity.com/paper/2110.04392/) (2021)
- [Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors](https://mlsanity.com/paper/2310.16924/) (2023)
- [Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT](https://mlsanity.com/paper/1802.06041/) (2018)
- [Unmet Needs and Opportunities for Mobile Translation AI](https://mlsanity.com/paper/2002.12387/) (2020)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answering](https://mlsanity.com/paper/2504.07583/) (2025)
- [Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions](https://mlsanity.com/paper/2502.13780/) (2025)
- [Facilitating Global Team Meetings Between Language-Based Subgroups: When and How Can Machine Translation Help?](https://mlsanity.com/paper/2209.02906/) (2022)
- [AskQE: Question Answering as Automatic Evaluation for Machine Translation](https://mlsanity.com/paper/2504.11582/) (2025)
- [Sustaining Human Agency, Attending to Its Cost: An Investigation into Generative AI Design for Non-Native Speakers' Language Use](https://mlsanity.com/paper/2503.07970/) (2025)
- [Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation](https://mlsanity.com/paper/2505.24683/) (2025)
- [(Dis)placed Contributions: Uncovering Hidden Hurdles to Collaborative Writing Involving Non-Native Speakers, Native Speakers, and AI-Powered Editing Tools](https://mlsanity.com/paper/2405.05474/) (2024)

## Built on by (most cited first, 1 of 1)

- [Measuring User's Mental Models of Speech Translation in Human-AI Collaboration](https://mlsanity.com/paper/2606.24644/) (2026)

## Abstract (arXiv)

As Machine Translation (MT) becomes increasingly commonplace, understanding how the general public perceives and relies on imperfect MT is crucial for contextualizing MT research in real-world applications. We present a human study conducted in a public museum (n=452), investigating how fluency and adequacy errors impact bilingual and non-bilingual users' reliance on MT during casual use. Our findings reveal that non-bilingual users often over-rely on MT due to a lack of evaluation strategies and alternatives, while experiencing the impact of errors can prompt users to reassess future reliance. This highlights the need for MT evaluation and NLP explanation techniques to promote not only MT quality, but also MT literacy among its users.