ixi-GEN: Efficient Industrial sLLMs through Domain Adaptive Continual Pretraining
Not on the map. Why?
Abstract (arXiv)
The emergence of open-source large language models (LLMs) has expanded opportunities for enterprise applications; however, many organizations still lack the infrastructure to deploy and maintain large-scale models. As a result, small LLMs (sLLMs) have become a practical alternative despite inherent performance limitations. While Domain Adaptive Continual Pretraining (DACP) has been explored for domain adaptation, its utility in commercial settings remains under-examined. In this study, we validate the effectiveness of a DACP-based recipe across diverse foundation models and service domains, producing DACP-applied sLLMs (ixi-GEN). Through extensive experiments and real-world evaluations, we demonstrate that ixi-GEN models achieve substantial gains in target-domain performance while preserving general capabilities, offering a cost-efficient and scalable solution for enterprise-level deployment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 9 Jul 2025.
Builds on
30 of 51 references.
References · 30
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2016Learning without Forgettingcited by 2,496
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2023BloombergGPT: A Large Language Model for Financecited by 939
- 2019Three scenarios for continual learningcited by 614
- 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
- 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
18 more
- 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
- 2022Fine-tuned Language Models are Continual Learnerscited by 156
- 2024How do Large Language Models Handle Multilingualism?cited by 150
- 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 137
- 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
- 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 119
- 2024Small Language Models: Survey, Measurements, and Insightscited by 112
- 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 95
- 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
- 2019Learning and Evaluating General Linguistic Intelligencecited by 79
- 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilitiescited by 69
- 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
- 2022KOBEST: Korean Balanced Evaluation of Significant Taskscited by 37
- 2024More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMscited by 21
- 2024From Text to Transformation: A Comprehensive Review of Large Language Models' Versatilitycited by 19
- 2024Improving Domain Adaptation through Extended-Text Reading Comprehensioncited by 10
- 2024Exploring Forgetting in Large Language Model Pre-Trainingcited by 9
- 2024Small Language Models for Application Interactions: A Case Studycited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026ixi-sLM: On-Device sLM for AI Call Agent Servicecited by 0
- 2026ixi-sLM: On-Device sLM for AI Call Agent Service14 Oct 2026