# Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing

arXiv: https://arxiv.org/abs/2101.03289 · HTML: https://mlsanity.com/paper/2101.03289/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 9 Jan 2021.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Deep Biaffine Attention for Neural Dependency Parsing](https://mlsanity.com/paper/1611.01734/) (2016)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks](https://mlsanity.com/paper/1903.05987/) (2019)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [BERTje: A Dutch BERT Model](https://mlsanity.com/paper/1912.09582/) (2019)
- [75 Languages, 1 Model: Parsing Universal Dependencies Universally](https://mlsanity.com/paper/1904.02099/) (2019)
- [UDapter: Language Adaptation for Truly Universal Dependency Parsing](https://mlsanity.com/paper/2004.14327/) (2020)
- [Small and Practical BERT Models for Sequence Labeling](https://mlsanity.com/paper/1909.00100/) (2019)
- [BERT Meets Chinese Word Segmentation](https://mlsanity.com/paper/1909.09292/) (2019)
- [Cross-lingual Word Sense Disambiguation using mBERT Embeddings with Syntactic Dependencies](https://mlsanity.com/paper/2012.05300/) (2020)
- [Cross-Lingual Adaptation Using Universal Dependencies](https://mlsanity.com/paper/2003.10816/) (2020)
- [Evaluating Multilingual BERT for Estonian](https://mlsanity.com/paper/2010.00454/) (2020)

## Built on by (most cited first, 30 of 68)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.16039/) (2023)
- [A Primer on Pretrained Multilingual Language Models](https://mlsanity.com/paper/2107.00676/) (2021)
- [A Closer Look at Claim Decomposition](https://mlsanity.com/paper/2403.11903/) (2024)
- [AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With](https://mlsanity.com/paper/2104.04052/) (2021)
- [MaLA-500: Massive Language Adaptation of Large Language Models](https://mlsanity.com/paper/2401.13303/) (2024)
- [GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation](https://mlsanity.com/paper/2306.01966/) (2023)
- [MEE: A Novel Multilingual Event Extraction Dataset](https://mlsanity.com/paper/2211.05955/) (2022)
- [Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modeling](https://mlsanity.com/paper/2311.18711/) (2023)
- [SlovakBERT: Slovak Masked Language Model](https://mlsanity.com/paper/2109.15254/) (2021)
- [A Second Wave of UD Hebrew Treebanking and Cross-Domain Parsing](https://mlsanity.com/paper/2210.07873/) (2022)
- [FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction](https://mlsanity.com/paper/2202.08316/) (2022)
- [MINION: a Large-Scale and Diverse Dataset for Multilingual Event Detection](https://mlsanity.com/paper/2211.05958/) (2022)
- [CLASSLA-Stanza: The Next Step for Linguistic Processing of South Slavic Languages](https://mlsanity.com/paper/2308.04255/) (2023)
- [The ParlaSent Multilingual Training Dataset for Sentiment Identification in Parliamentary Proceedings](https://mlsanity.com/paper/2309.09783/) (2023)
- [Probing Simile Knowledge from Pre-trained Language Models](https://mlsanity.com/paper/2204.12807/) (2022)
- [Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew](https://mlsanity.com/paper/2311.00658/) (2023)
- [On the Relation between Syntactic Divergence and Zero-Shot Performance](https://mlsanity.com/paper/2110.04644/) (2021)
- [Morphological Processing of Low-Resource Languages: Where We Are and What's Next](https://mlsanity.com/paper/2203.08909/) (2022)
- [A Novel Multi-Task Learning Approach for Context-Sensitive Compound Type Identification in Sanskrit](https://mlsanity.com/paper/2208.10310/) (2022)
- [Improving Cross-Lingual Transfer through Subtree-Aware Word Reordering](https://mlsanity.com/paper/2310.13583/) (2023)
- [Code-Switching with Word Senses for Pretraining in Neural Machine Translation](https://mlsanity.com/paper/2310.14050/) (2023)
- [A Truly Joint Neural Architecture for Segmentation and Parsing](https://mlsanity.com/paper/2402.02564/) (2024)
- [The Knesset Corpus: An Annotated Corpus of Hebrew Parliamentary Proceedings](https://mlsanity.com/paper/2405.18115/) (2024)
- [Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations](https://mlsanity.com/paper/2407.04543/) (2024)
- [Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation](https://mlsanity.com/paper/2507.07572/) (2025)
- [TGIF: Tree-Graph Integrated-Format Parser for Enhanced UD with Two-Stage Generic- to Individual-Language Finetuning](https://mlsanity.com/paper/2107.06907/) (2021)
- [Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing](https://mlsanity.com/paper/2201.11374/) (2022)
- [HeartBEiT: Vision Transformer for Electrocardiogram Data Improves Diagnostic Performance at Low Sample Sizes](https://mlsanity.com/paper/2212.14040/) (2022)

## Abstract (arXiv)

We introduce Trankit, a light-weight Transformer-based Toolkit for multilingual Natural Language Processing (NLP). It provides a trainable pipeline for fundamental NLP tasks over 100 languages, and 90 pretrained pipelines for 56 languages. Built on a state-of-the-art pretrained language model, Trankit significantly outperforms prior multilingual NLP pipelines over sentence segmentation, part-of-speech tagging, morphological feature tagging, and dependency parsing while maintaining competitive performance for tokenization, multi-word token expansion, and lemmatization over 90 Universal Dependencies treebanks. Despite the use of a large pretrained transformer, our toolkit is still efficient in memory usage and speed. This is achieved by our novel plug-and-play mechanism with Adapters where a multilingual pretrained transformer is shared across pipelines for different languages. Our toolkit along with pretrained models and code are publicly available at: https://github.com/nlp-uoregon/trankit. A demo website for our toolkit is also available at: http://nlp.uoregon.edu/trankit. Finally, we create a demo video for Trankit at: https://youtu.be/q0KGP3zGjGc.