# fugashi, a Tool for Tokenizing Japanese in Python

arXiv: https://arxiv.org/abs/2010.06858 · HTML: https://mlsanity.com/paper/2010.06858/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 17 of 17)

- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [Revisiting non-English Text Simplification: A Unified Multilingual Benchmark](https://mlsanity.com/paper/2305.15678/) (2023)
- [The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any language](https://mlsanity.com/paper/2311.08323/) (2023)
- [Confounding Factors in Relating Model Performance to Morphology](https://mlsanity.com/paper/2511.01380/) (2025)
- [Large Scale Multi-Lingual Multi-Modal Summarization Dataset](https://mlsanity.com/paper/2302.06560/) (2023)
- [Against All Odds: Overcoming Typology, Script, and Language Confusion in Multilingual Embedding Inversion Attacks](https://mlsanity.com/paper/2408.11749/) (2024)
- [Parameterized Synthetic Text Generation with SimpleStories](https://mlsanity.com/paper/2504.09184/) (2025)
- [Combining Static and Contextualised Multilingual Embeddings](https://mlsanity.com/paper/2203.09326/) (2022)
- [Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis](https://mlsanity.com/paper/2410.13237/) (2024)
- [IndexTTS 2.5 Technical Report](https://mlsanity.com/paper/2601.03888/) (2026)
- [cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation](https://mlsanity.com/paper/2206.03354/) (2022)
- [A Cascade Model for Argument Mining in Japanese Political Discussions: the QA Lab-PoliInfo-3 Case Study](https://mlsanity.com/paper/2207.01672/) (2022)
- [CinPatent: Datasets for Patent Classification](https://mlsanity.com/paper/2212.12192/) (2022)
- [FewShotTextGCN: K-hop neighborhood regularization for few-shot learning on graphs](https://mlsanity.com/paper/2301.10481/) (2023)
- [The Effects of Political Martyrdom on Election Results: The Assassination of Abe](https://mlsanity.com/paper/2305.18004/) (2023)
- [CONCAP: Seeing Beyond English with Concepts Retrieval-Augmented Captioning](https://mlsanity.com/paper/2507.20411/) (2025)
- [TalkMatrix: Generating Character Dialogue that is Both Consistent and Diverse](https://mlsanity.com/paper/2609.19022/) (2026)

## Abstract (arXiv)

Recent years have seen an increase in the number of large-scale multilingual NLP projects. However, even in such projects, languages with special processing requirements are often excluded. One such language is Japanese. Japanese is written without spaces, tokenization is non-trivial, and while high quality open source tokenizers exist they can be hard to use and lack English documentation. This paper introduces fugashi, a MeCab wrapper for Python, and gives an introduction to tokenizing Japanese.