mlsanityData to 14 Oct 2026

LongCat-Flash Technical Report

arXiv 2509.01322 ↗First posted 1 Sep 2025cs.CL cs.AI cs.DCcited by 58

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming from the need for scalable efficiency, LongCat-Flash adopts two novel designs: (a) Zero-computation Experts, which enables dynamic computational budget allocation and activates 18.6B-31.3B (27B on average) per token depending on contextual demands, optimizing resource usage. (b) Shortcut-connected MoE, which enlarges the computation-communication overlap window, demonstrating notable gains in inference efficiency and throughput compared to models of a comparable scale. We develop a comprehensive scaling framework for large models that combines hyperparameter transfer, model-growth initialization, a multi-pronged stability suite, and deterministic computation to achieve stable and reproducible training. Notably, leveraging the synergy among scalable architectural design and infrastructure efforts, we complete model training on more than 20 trillion tokens within 30 days, while achieving over 100 tokens per second (TPS) for inference at a cost of 0.70 per million output tokens. To cultivate LongCat-Flash towards agentic intelligence, we conduct a large-scale pre-training on optimized mixtures, followed by targeted mid- and post-training on reasoning, code, and instructions, with further augmentation from synthetic data and tool use tasks. Comprehensive evaluations demonstrate that, as a non-thinking foundation model, LongCat-Flash delivers highly competitive performance among other leading models, with exceptional strengths in agentic tasks. The model checkpoint of LongCat-Flash is open-sourced to foster community research. LongCat Chat: https://longcat.ai Hugging Face: https://huggingface.co/meituan-longcat GitHub: https://github.com/meituan-longcat

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 71 of its 100 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

71 of 100 references.

References · 71

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  12. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
38 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  7. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  8. 2019Root Mean Square Layer Normalizationcited by 1,480
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  12. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  13. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  14. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  15. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  16. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  17. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  18. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  19. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  20. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  21. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 399
  22. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  23. 20242 OLMo 2 Furiouscited by 340
  24. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
  25. 2024Massive Activations in Large Language Modelscited by 294
  26. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  27. 2015Net2Net: Accelerating Learning via Knowledge Transfercited by 250
  28. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 205
  29. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
  30. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 193
  31. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  32. 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 136
  33. 2024Rule Based Rewards for Language Model Safetycited by 108
  34. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decodingcited by 91
  35. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughputcited by 90
  36. 2023Learning to Grow Pretrained Models for Efficient Transformer Trainingcited by 83
  37. 2024Scaling Exponents Across Parameterizations and Optimizerscited by 76
  38. 2024Evaluating Language Models for Efficient Code Generationcited by 73

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 42.

58 papers.

Sort:

By area:

  1. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
  2. 2025VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applicationscited by 53
  3. 2026LongCat-Flash-Thinking-2601 Technical Reportcited by 34
  4. 2025LongCat-Flash-Omni Technical Reportcited by 26
  5. 2026AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Conditioncited by 23
  6. 2025Introducing LongCat-Flash-Thinking: A Technical Reportcited by 22
  7. 2025How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretrainingcited by 14
  8. 2026DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Trainingcited by 14
  9. 2025FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Predictioncited by 13
  10. 2026Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Textcited by 13
  11. 2025Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactionscited by 12
  12. 2025ADHint: Adaptive Hints with Difficulty Priors for Reinforcement Learningcited by 12
38 more
  1. 2026VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactionscited by 12
  2. 2026HiFloat4 Format for Language Model Inferencecited by 11
  3. 2026Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigationcited by 11
  4. 2025Towards a Comprehensive Scaling Law of Mixture-of-Expertscited by 10
  5. 2025Higher Satisfaction, Lower Cost: A Technical Report on How LLMs Revolutionize Meituan's Intelligent Interaction Systemscited by 7
  6. 2025MoE Pathfinder: Trajectory-driven Expert Pruningcited by 7
  7. 2026XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compressioncited by 7
  8. 2026V0: A Generalist Value Model for Any Policy at State Zerocited by 7
  9. 2026Fast Catch-Up, Late Switching: Optimal Batch Size Scheduling via Functional Scaling Lawscited by 6
  10. 2026General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Taskscited by 6
  11. 2026Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environmentscited by 6
  12. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  13. 2026FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluationcited by 5
  14. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyondcited by 5
  15. 2026SnapMLA: Efficient Long-Context MLA Decoding via Hardware-Aware FP8 Quantized Pipeliningcited by 4
  16. 2026Post-Trained MoE Can Skip Half Experts via Self-Distillationcited by 4
  17. 2026APPO: Agentic Procedural Policy Optimizationcited by 4
  18. 2026CirrusBench: Evaluating LLM-based Agents Beyond Correctness in Real-World Cloud Service Environmentscited by 3
  19. 2026LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budgetcited by 3
  20. 2026SpanNorm: Reconciling Training Stability and Performance in Deep Transformerscited by 2
  21. 2026Multi-Head Low-Rank Attentioncited by 2
  22. 2026ACE: Pluggable Adaptive Context Elasticizer across Agentscited by 2
  23. 2026LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexingcited by 2
  24. 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateauscited by 2
  25. 2025SELF-REDRAFT: Eliciting Intrinsic Exploration-Exploitation Balance in Test-Time Scaling for Code Generationcited by 1
  26. 2025Scaling and Transferability of Annealing Strategies in Large Language Model Trainingcited by 1
  27. 2026HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-benchcited by 1
  28. 2026FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Controlcited by 1
  29. 2026Reinforcing Real-world Service Agents: Balancing Utility and Cost in Task-oriented Dialoguecited by 1
  30. 2026Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Modelscited by 1
  31. 2025TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefixcited by 0
  32. 2025AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expertcited by 0
  33. 2025AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealingcited by 0
  34. 2026OPE: Overcoming Information Saturation in Parallel Thinking via Outline-Guided Path Explorationcited by 0
  35. 2026XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compressioncited by 0
  36. 2026FineRMoE: Dimension Expansion for Finer-Grained Expert with Its Upcycling Approachcited by 0
  37. 2026Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modelingcited by 0
  38. 2026DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Schedulingcited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters
  2. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
  3. 2025MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
  4. 2026JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency
  5. 2026LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
  6. 2026Yuan3.0 Flash: An Open Multimodal Large Language Model for Enterprise Applications
  7. 2026Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
  8. 2025Faster MoE LLM Inference for Extremely Large Models