Vocabulary Customization for Efficient Domain-Specific LLM Deployment
Not on the map. Why?
Abstract (arXiv)
When using an LLM to process text outside the training domain(s), an often overlooked factor is vocabulary mismatch, where the general-domain tokenizer fails to capture frequent domain-specific terms, leading to higher token fertility and thus a decrease in processing speed due to suboptimal sub-word splits. We address this limitation by augmenting the pretrained vocabulary with a set of domain-specific tokens. To this end, we design an algorithm that extends an existing tokenizer while guaranteeing it never decreases tokenization efficiency: every input sequence is segmented into at most the same number of tokens as before. Evaluated on real-world e-Commerce use-cases, the augmented tokenizer significantly shortens input sequences by up to 20% and reduces inference latency on downstream tasks while preserving predictive quality. We further analyze secondary effects, such as the impact on forward pass speed and the rate at which the model adopts the newly introduced tokens, to illustrate the broader benefits of vocabulary adaptation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 50 of its 56 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
50 of 56 references.
References · 50
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2023Code Llama: Open Foundation Models for Codecited by 2,105
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
38 more
- 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
- 2023Llemma: An Open Language Model For Mathematicscited by 385
- 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
- 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
- 2021Block Pruning For Faster Transformerscited by 176
- 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
- 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
- 2024Getting the most out of your tokenizer for pre-training and domain adaptationcited by 65
- 2024ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Changecited by 46
- 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 44
- 2024Fast Vocabulary Transfer for Language Model Compressioncited by 42
- 2021Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domainscited by 36
- 2024Mitigating Catastrophic Forgetting in Language Transfer via Model Mergingcited by 36
- 2024Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPcited by 35
- 2021Efficient Domain Adaptation of Language Models via Adaptive Tokenizationcited by 34
- 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
- 2023Tamil-Llama: A New Tamil Language Model Based on Llama 2cited by 31
- 2023Efficiently Adapting Pretrained Language Models To New Languagescited by 28
- 2023Typhoon: Thai Large Language Modelscited by 28
- 2024MaLA-500: Massive Language Adaptation of Large Language Modelscited by 28
- 2023Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learningcited by 26
- 2023Cabrita: closing the gap for foreign languagescited by 25
- 2020Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QAcited by 23
- 2024How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?cited by 22
- 2024Exploring Design Choices for Building Language-Specific LLMscited by 22
- 2024An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Modelscited by 20
- 2024An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inferencecited by 18
- 2025AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptationcited by 15
- 2021Fine-Tuning Transformers: Vocabulary Transfercited by 14
- 2024Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Koreancited by 14
- 2022CINO: A Chinese Minority Pre-trained Language Modelcited by 11
- 2024Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMscited by 11
- 2025Domain Adaptation of Foundation LLMs for e-Commercecited by 8
- 2024Investigating LLM Applications in E-Commercecited by 7
- 2024Adapting Chat Language Models Using Only Target Unlabeled Language Datacited by 6
- 2024ReTok: Replacing Tokenizer to Enhance Representation Efficiency in Large Language Modelcited by 5
- 2024IKUN for WMT24 General MT Task: LLMs Are here for Multilingual Machine Translationcited by 3
- 2024Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enoughcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.