mlsanityData to 7 Oct 2026

A.X K1 Technical Report

arXiv 2601.09200 ↗First posted 14 Jan 2026cs.CL cs.AIcited by 0

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

We introduce A.X K1, a 519B-parameter Mixture-of-Experts (MoE) language model trained from scratch. Our design leverages scaling laws to optimize training configurations and vocabulary size under fixed computational budgets. A.X K1 is pre-trained on a corpus of approximately 10T tokens, curated by a multi-stage data processing pipeline. Designed to bridge the gap between reasoning capability and inference efficiency, A.X K1 supports explicitly controllable reasoning to facilitate scalable deployment across diverse real-world scenarios. We propose a simple yet effective Think-Fusion training recipe, enabling user-controlled switching between thinking and non-thinking modes within a single unified model. Extensive evaluations demonstrate that A.X K1 achieves performance competitive with leading open-source models, while establishing a distinctive advantage in Korean-language benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 30 of its 42 references that are in the map. First posted on arXiv 14 Jan 2026.

Builds on

30 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Models23 Jul

Other references · 29

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2025Gemma 3 Technical Reportcited by 1,686
  12. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
17 more
  1. 2019Root Mean Square Layer Normalizationcited by 1,425
  2. 2025s1: Simple test-time scalingcited by 1,311
  3. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  7. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  8. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  9. 2025Humanity's Last Examcited by 467
  10. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  11. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 300
  12. 2024Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabulariescited by 113
  13. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  14. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  15. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 41
  16. 2025MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologiescited by 23
  17. 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?cited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A.X K2 Technical Report
  2. 2025FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
  3. 2024A Closer Look into Mixture-of-Experts in Large Language Models
  4. 2026DAG-MoE: From Simple Mixture to Structural Aggregation in Mixture-of-Experts
  5. 2024Toward Inference-optimal Mixture-of-Expert Large Language Models
  6. 2025dots.llm1 Technical Report
  7. 2025KORMo: Korean Open Reasoning Model for Everyone
  8. 2026K-EXAONE 2.0 Technical Report