mlsanityData to 14 Oct 2026

Survey of Specialized Large Language Model

arXiv 2508.19667 ↗First posted 27 Aug 2025cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

The rapid evolution of specialized large language models (LLMs) has transitioned from simple domain adaptation to sophisticated native architectures, marking a paradigm shift in AI development. This survey systematically examines this progression across healthcare, finance, legal, and technical domains. Besides the wide use of specialized LLMs, technical breakthrough such as the emergence of domain-native designs beyond fine-tuning, growing emphasis on parameter efficiency through sparse computation and quantization, increasing integration of multimodal capabilities and so on are applied to recent LLM agent. Our analysis reveals how these innovations address fundamental limitations of general-purpose LLMs in professional applications, with specialized models consistently performance gains on domain-specific benchmarks. The survey further highlights the implications for E-Commerce field to fill gaps in the field.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

28 of 43 references.

References · 28

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  5. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  7. 2023BloombergGPT: A Large Language Model for Financecited by 939
  8. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  10. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  11. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  12. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
16 more
  1. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  2. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  3. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 195
  4. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 156
  5. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
  6. 2021Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inferencecited by 96
  7. 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 92
  8. 2023System 2 Attention (is something you might need too)cited by 89
  9. 2023InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuningcited by 82
  10. 2023ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Textscited by 78
  11. 2025ECom-Bench: Can LLM Agent Resolve Real-World E-commerce Customer Support Issues?cited by 27
  12. 2025Accurate KV Cache Quantization with Outlier Tokens Tracingcited by 18
  13. 2025MindFlow: Revolutionizing E-commerce Customer Support with Multimodal LLM Agentscited by 6
  14. 2025MeLA: A Metacognitive LLM-Driven Architecture for Automatic Heuristic Designcited by 4
  15. 2025Quantifying the Importance of Data Alignment in Downstream Model Performancecited by 2
  16. 2025FinBERT-QA: Financial Question Answering with pre-trained BERT Language Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Trainingcited by 4
  2. 2026CGRA-DeBERTa Concept Guided Residual Augmentation Transformer for Theologically Islamic Understandingcited by 0
  3. 2026Recovering General Capabilities via Uncertainty-Calibrated Multi-Teacher On-Policy Distillationcited by 0
  4. 2026HuatuoGPT-3: RL-Only Domain Adaptation from Base Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Large Language Models with some Insights on their Capabilities and Limitations
  2. 2025LLM-Powered AI Agent Systems and Their Applications in Industry
  3. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  5. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Review
  6. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Models
  7. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results
  8. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects