# Machine Translation Evaluation Benchmark for Wu Chinese: Workflow and Analysis

arXiv: https://arxiv.org/abs/2410.10278 · HTML: https://mlsanity.com/paper/2410.10278/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 7 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 3 of 3)

- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Improving TTS for Shanghainese: Addressing Tone Sandhi via Word Segmentation](https://mlsanity.com/paper/2307.16199/) (2023)

## Built on by (most cited first, 2 of 2)

- [SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages](https://mlsanity.com/paper/2509.20557/) (2025)
- [HardMTBench: Stress-Testing Chinese-English Translation on Knowledge-Intensive Domains](https://mlsanity.com/paper/2605.28315/) (2026)

## Abstract (arXiv)

We introduce a FLORES+ dataset as an evaluation benchmark for modern Wu Chinese machine translation models and showcase its compatibility with existing Wu data. Wu Chinese is mutually unintelligible with other Sinitic languages such as Mandarin and Yue (Cantonese), but uses a set of Hanzi (Chinese characters) that profoundly overlaps with others. The population of Wu speakers is the second largest among languages in China, but the language has been suffering from significant drop in usage especially among the younger generations. We identify Wu Chinese as a textually low-resource language and address challenges for its machine translation models. Our contributions include: (1) an open-source, manually translated dataset, (2) full documentations on the process of dataset creation and validation experiments, (3) preliminary tools for Wu Chinese normalization and segmentation, and (4) benefits and limitations of our dataset, as well as implications to other low-resource languages.