# Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slip Scripts

arXiv: https://arxiv.org/abs/2409.01011 · HTML: https://mlsanity.com/paper/2409.01011/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning](https://mlsanity.com/paper/1910.09433/) (2019)
- [Radical-Enhanced Chinese Character Embedding](https://mlsanity.com/paper/1404.4714/) (2014)
- [Component-Enhanced Chinese Character Embeddings](https://mlsanity.com/paper/1508.06669/) (2015)
- [Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling](https://mlsanity.com/paper/2211.12781/) (2022)
- [Knowledge of Pretrained Language Models on Surface Information of Tokens](https://mlsanity.com/paper/2402.09808/) (2024)

## Built on by (most cited first, 1 of 1)

- [Towards Computational Chinese Paleography](https://mlsanity.com/paper/2601.06753/) (2026)

## Abstract (arXiv)

This study presents a multi-modal multi-granularity tokenizer specifically designed for analyzing ancient Chinese scripts, focusing on the Chu bamboo slip (CBS) script used during the Spring and Autumn and Warring States period (771-256 BCE) in Ancient China. Considering the complex hierarchical structure of ancient Chinese scripts, where a single character may be a combination of multiple sub-characters, our tokenizer first adopts character detection to locate character boundaries, and then conducts character recognition at both the character and sub-character levels. Moreover, to support the academic community, we have also assembled the first large-scale dataset of CBSs with over 100K annotated character image scans. On the part-of-speech tagging task built on our dataset, using our tokenizer gives a 5.5% relative improvement in F1-score compared to mainstream sub-word tokenizers. Our work not only aids in further investigations of the specific script but also has the potential to advance research on other forms of ancient Chinese scripts.