mlsanityData to 14 Oct 2026

ixi-GEN: Efficient Industrial sLLMs through Domain Adaptive Continual Pretraining

arXiv 2507.06795 ↗First posted 9 Jul 2025cs.CL cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

The emergence of open-source large language models (LLMs) has expanded opportunities for enterprise applications; however, many organizations still lack the infrastructure to deploy and maintain large-scale models. As a result, small LLMs (sLLMs) have become a practical alternative despite inherent performance limitations. While Domain Adaptive Continual Pretraining (DACP) has been explored for domain adaptation, its utility in commercial settings remains under-examined. In this study, we validate the effectiveness of a DACP-based recipe across diverse foundation models and service domains, producing DACP-applied sLLMs (ixi-GEN). Through extensive experiments and real-world evaluations, we demonstrate that ixi-GEN models achieve substantial gains in target-domain performance while preserving general capabilities, offering a cost-efficient and scalable solution for enterprise-level deployment.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

30 of 51 references.

References · 30

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2016Learning without Forgettingcited by 2,496
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  8. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  9. 2023BloombergGPT: A Large Language Model for Financecited by 939
  10. 2019Three scenarios for continual learningcited by 614
  11. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  12. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
18 more
  1. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  2. 2022Fine-tuned Language Models are Continual Learnerscited by 156
  3. 2024How do Large Language Models Handle Multilingualism?cited by 150
  4. 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 137
  5. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  6. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 119
  7. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
  8. 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 95
  9. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  10. 2019Learning and Evaluating General Linguistic Intelligencecited by 79
  11. 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilitiescited by 69
  12. 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
  13. 2022KOBEST: Korean Balanced Evaluation of Significant Taskscited by 37
  14. 2024More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMscited by 21
  15. 2024From Text to Transformation: A Comprehensive Review of Large Language Models' Versatilitycited by 19
  16. 2024Improving Domain Adaptation through Extended-Text Reading Comprehensioncited by 10
  17. 2024Exploring Forgetting in Large Language Model Pre-Trainingcited by 9
  18. 2024Small Language Models for Application Interactions: A Case Studycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.