mlsanityData to 7 Oct 2026

Is Micro Domain-Adaptive Pre-Training Effective for Real-World Operations? Multi-Step Evaluation Reveals Potential and Bottlenecks

arXiv 2602.04466 ↗First posted 4 Feb 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

When applying LLMs to real-world enterprise operations, LLMs need to handle proprietary knowledge in small domains of specific operations (micro domains). A previous study shows micro domain-adaptive pre-training (mDAPT) with fewer documents is effective, similarly to DAPT in larger domains. However, it evaluates mDAPT only on multiple-choice questions; thus, its effectiveness for generative tasks in real-world operations remains unknown. We aim to reveal the potential and bottlenecks of mDAPT for generative tasks. To this end, we disentangle the answering process into three subtasks and evaluate the performance of each subtask: (1) eliciting facts relevant to questions from an LLM's own knowledge, (2) reasoning over the facts to obtain conclusions, and (3) composing long-form answers based on the conclusions. We verified mDAPT on proprietary IT product knowledge for real-world questions in IT technical support operations. As a result, mDAPT resolved the elicitation task that the base model struggled with but did not resolve other subtasks. This clarifies mDAPT's effectiveness in the knowledge aspect and its bottlenecks in other aspects. Further analysis empirically shows that resolving the elicitation and reasoning tasks ensures sufficient performance (over 90%), emphasizing the need to enhance reasoning capability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 25 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

15 of 25 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  6. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  7. 2023BloombergGPT: A Large Language Model for Financecited by 923
  8. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  9. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 105
  10. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  11. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  12. 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
3 more
  1. 2024Instruction-tuned Language Models are Better Knowledge Learnerscited by 54
  2. 2024CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentationcited by 14
  3. 2025Agent Fine-tuning through Distillation for Domain-specific LLMs in Microdomainscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023When Giant Language Brains Just Aren't Enough! Domain Pizzazz with Knowledge Sparkle Dust
  2. 2024Hybrid Training Approaches for LLMs: Leveraging Real and Synthetic Data to Enhance Model Performance in Domain-Specific Applications
  3. 2024Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs
  4. 2025Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
  5. 2024Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?
  6. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains
  7. 2024Evaluating LLM Reasoning in the Operations Research Domain with ORQA
  8. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models