mlsanityData to 7 Oct 2026

Kimi K3: Open Frontier Intelligence

arXiv 2607.24653 ↗First posted 27 Jul 2026cs.CL cs.LGcited by 99

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 85 of its 125 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

85 of 125 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters11 Feb
  2. 2026APEX-Agents20 Jan
  3. 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution29 Oct

Other references · 82

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  10. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  11. 2020GLU Variants Improve Transformercited by 1,724
  12. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
38 more
  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  2. 2019Root Mean Square Layer Normalizationcited by 1,425
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  4. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  5. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  7. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  8. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  10. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  11. 2016Language Modeling with Gated Convolutional Networkscited by 688
  12. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  13. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  14. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  15. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  16. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  17. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  18. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  19. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  20. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  21. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  22. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  23. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  24. 2025Muon is Scalable for LLM Trainingcited by 381
  25. 2025Kimi-VL Technical Reportcited by 381
  26. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  27. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 315
  28. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  29. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  30. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 232
  31. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  32. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 172
  33. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 169
  34. 2026MiMo-V2-Flash Technical Reportcited by 163
  35. 2018Online normalizer calculation for softmaxcited by 154
  36. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 144
  37. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
  38. 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 134

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 99.

99 papers.

Sort:

By area:

  1. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 15
  2. 2026Stealing Reasoning Traces from Proprietary LLM APIscited by 8
  3. 2026Rethinking Privileged Information in On-Policy Self-Distillationcited by 6
  4. 2026Extremely Sparse Supervision Incentivizes Reasoning Abilitycited by 3
  5. 2026Multimodal Duplex Interaction Agentcited by 3
  6. 2026EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learningcited by 2
  7. 2026Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirementscited by 2
  8. 2026DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decodingcited by 2
  9. 2026Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillationcited by 2
  10. 2026Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commercecited by 1
  11. 2026Scheduling Mixed RL Rollouts Beyond Prefix Localitycited by 1
  12. 2026Language Models Can Control Their Own Attentioncited by 1
38 more
  1. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusioncited by 1
  2. 2026Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flatteningcited by 1
  3. 2026Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluationscited by 1
  4. 2026RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learningcited by 1
  5. 2026Groupwise Agentic Grading and Advantage Redistribution for Code Agent RLcited by 1
  6. 2026From BERT to Frontier Agents: Eight Years of Language-Model Progress, the Collapse of the Capability-Cost Curve, and the Rise of Task-Targeted Modelscited by 0
  7. 2026SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary systemcited by 0
  8. 2026On the Principles Behind Neural Network Optimizerscited by 0
  9. 2026How Much Rank Does LoRA Need? Rank-Error Bounds for Transformer Attentioncited by 0
  10. 2026TuringLLM: Efficiently Scaling Foundation Models Toward Physical AIcited by 0
  11. 2026The Answer Is Not the Argumentcited by 0
  12. 2026Instella-MoE Technical Reportcited by 0
  13. 2026Towards a Statistical Understanding of Mixture-of-Expertscited by 0
  14. 2026RealCADBench: Benchmarking Parametric CAD Modeling from Industrial Design Intentscited by 0
  15. 2026VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identificationcited by 0
  16. 2026CantoneseLLM v2: Reasoning in a Low-Resource Languagecited by 0
  17. 2026Aha-Flow Distillation: Flow Markers Matter in LLM Reasoningcited by 0
  18. 2026xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problemscited by 0
  19. 2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Modelscited by 0
  20. 2026Hyperparameter Scaling Laws Across MoE Sparsitycited by 0
  21. 2026SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?cited by 0
  22. 2026IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifiercited by 0
  23. 2026Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verificationcited by 0
  24. 2026Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solvingcited by 0
  25. 2026ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Searchcited by 0
  26. 2026The Token Before the Value Is the Key: How Hybrid Architectures Organize Induction Circuitscited by 0
  27. 2026CHART: A Harness-Rotation Curriculum for Harness-Robust Search Agentscited by 0
  28. 2026AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulationcited by 0
  29. 2026Opinion Leader Dynamics: How Sparse Attention Shapes Token Clusteringcited by 0
  30. 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 0
  31. 2026FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceabilitycited by 0
  32. 2026WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learningcited by 0
  33. 2026SWE-Serve: Benchmarking Agentic Engineering For Production Inference Servingcited by 0
  34. 2026When Context Misleads: In-context Learning with Jurisdiction in Large Language Modelscited by 0
  35. 2026AdaHVLA: Adaptive Harnesses for Long-Horizon Vision-Language-Action Executioncited by 0
  36. 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
  37. 2026Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Modelscited by 0
  38. 2026SoFT: Soft Targets for Generalizable LLM Fine-Tuningcited by 0

50 most cited of 99 All 99 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Kimi K2: Open Agentic Intelligence
  2. 2026A.X K1 Technical Report
  3. 2026A.X K2 Technical Report
  4. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
  5. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
  6. 2026Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
  7. 2024Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
  8. 2025INTELLECT-3: Technical Report