# User-Centric Evaluation of OCR Systems for Kwak'wala

arXiv: https://arxiv.org/abs/2302.13410 · HTML: https://mlsanity.com/paper/2302.13410/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 30 references that are in the map.
- First posted on arXiv 26 Feb 2023.

## Builds on (most cited first, 5 of 5)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [OCR Post Correction for Endangered Language Texts](https://mlsanity.com/paper/2011.05402/) (2020)
- [Lexically Aware Semi-Supervised Learning for OCR Post-Correction](https://mlsanity.com/paper/2111.02622/) (2021)
- [Upcycle Your OCR: Reusing OCRs for Post-OCR Text Correction in Romanised Sanskrit](https://mlsanity.com/paper/1809.02147/) (2018)
- [OCR quality affects perceived usefulness of historical newspaper clippings -- a user study](https://mlsanity.com/paper/2203.03557/) (2022)

## Built on by (most cited first, 4 of 4)

- [GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text](https://mlsanity.com/paper/2403.06399/) (2024)
- [Comparative analysis of optical character recognition methods for S'ami texts from the National Library of Norway](https://mlsanity.com/paper/2501.07300/) (2025)
- [Massively Multilingual Joint Segmentation and Glossing](https://mlsanity.com/paper/2601.10925/) (2026)
- [Developing a Mixed-Methods Pipeline for Community-Oriented Digitization of Kwak'wala Legacy Texts](https://mlsanity.com/paper/2506.01775/) (2025)

## Abstract (arXiv)

There has been recent interest in improving optical character recognition (OCR) for endangered languages, particularly because a large number of documents and books in these languages are not in machine-readable formats. The performance of OCR systems is typically evaluated using automatic metrics such as character and word error rates. While error rates are useful for the comparison of different models and systems, they do not measure whether and how the transcriptions produced from OCR tools are useful to downstream users. In this paper, we present a human-centric evaluation of OCR systems, focusing on the Kwak'wala language as a case study. With a user study, we show that utilizing OCR reduces the time spent in the manual transcription of culturally valuable documents -- a task that is often undertaken by endangered language community members and researchers -- by over 50%. Our results demonstrate the potential benefits that OCR tools can have on downstream language documentation and revitalization efforts.