mlsanityData to 7 Oct 2026

Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia

arXiv 2509.09121 ↗First posted 11 Sep 2025cs.CLcited by 3

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Large language models (LLMs) excel in general-domain applications, yet their performance often degrades in specialized tasks requiring domain-specific knowledge. E-commerce is particularly challenging, as its data are noisy, heterogeneous, multilingual, and highly dynamic. We present Compass-v3, a vertical-domain Mixture-of-Experts (MoE) model with 245B total parameters and 71B active per token, designed for Southeast Asian e-commerce. Compass-v3 adopts fewer but larger experts, combined with hardware-efficient optimizations-such as intra-node expert parallelism and a customized memcpy operator-to maximize GPU utilization. The model is trained on 12T tokens of curated multilingual corpora and large-scale synthetic e-commerce instructions using a mixed-training strategy. To enhance alignment, we propose Optimal-Transport Direct Preference Optimization (OTPO), which captures token-level distinctions and improves instruction adherence in commerce-specific scenarios. Extensive evaluations demonstrate that Compass-v3 delivers state-of-the-art e-commerce performance, surpassing DeepSeek-V3.1, GPT-4 series, and Qwen3-235B. Moreover, Compass-v3 demonstrates strong multilingual capability across low-resource Southeast Asian languages (Indonesian, Thai, Filipino, Vietnamese, Malay, Taglog) and Portuguese while sustaining competitive performance on general benchmarks. It has already been widely applied in Shopee's industrial-scale e-commerce platform and is gradually replacing OpenAI's traffic, now accounting for over 70% of total LLM usage, highlighting its dual strengths in specialized commerce expertise and broad linguistic competence.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 22 of its 29 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

22 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Compass-V2 Technical Report22 Apr
  2. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Models17 Feb

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  8. 2023BloombergGPT: A Large Language Model for Financecited by 923
  9. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
  10. 2023Towards Generalist Biomedical AIcited by 286
  11. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  12. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
8 more
  1. 2025HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languagescited by 71
  2. 2023EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commercecited by 48
  3. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 44
  4. 2024Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretrainingcited by 24
  5. 2025QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM Pretrainingcited by 16
  6. 2024A survey on fairness of large language models in e-commerce: progress, application, and challengecited by 10
  7. 2023LLaMA-E: Empowering E-commerce Authoring with Object-Interleaved Instruction Followingcited by 7
  8. 2025Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimizationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Each Prompt Matters: Scaling Reinforcement Learning Without Wasting Rollouts on Hundred-Billion-Scale MoEcited by 2
  2. 2026Valley3: Scaling Omni Foundation Models for E-commercecited by 1
  3. 2025MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Aligning LLMs for Multilingual Consistency in Enterprise Applications
  2. 2024Compass: Large Multilingual Language Model for South-east Asia
  3. 2025ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models
  4. 2025Survey of Specialized Large Language Model
  5. 2025Orchestrating Heterogeneous Experts: A Scalable MoE Framework with Anisotropy-Preserving Fusion
  6. 2026EComStage: Stage-wise and Orientation-specific Benchmarking for Large Language Models in E-commerce
  7. 2026SIA: A Synthesize-Inject-Align Framework for Knowledge-Grounded and Secure E-commerce Search LLMs with Industrial Deployment
  8. 2024EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization