# Evaluating Transformer Models and Human Behaviors on Chinese Character Naming

arXiv: https://arxiv.org/abs/2303.12294 · HTML: https://mlsanity.com/paper/2303.12294/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 40 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection](https://mlsanity.com/paper/1906.01280/) (2019)
- [Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German plurals](https://mlsanity.com/paper/2005.08826/) (2020)

## Abstract (arXiv)

Neural network models have been proposed to explain the grapheme-phoneme mapping process in humans for many alphabet languages. These models not only successfully learned the correspondence of the letter strings and their pronunciation, but also captured human behavior in nonce word naming tasks. How would the neural models perform for a non-alphabet language (e.g., Chinese) unknown character task? How well would the model capture human behavior? In this study, we first collect human speakers' answers on unknown character naming tasks and then evaluate a set of transformer models by comparing their performances with human behaviors on an unknown Chinese character naming task. We found that the models and humans behaved very similarly, that they had similar accuracy distribution for each character, and had a substantial overlap in answers. In addition, the models' answers are highly correlated with humans' answers. These results suggested that the transformer models can well capture human's character naming behavior.