mlsanityData to 7 Oct 2026

Ruyi2 Technical Report

arXiv 2602.22543 ↗First posted 26 Feb 2026cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) face significant challenges regarding deployment costs and latency, necessitating adaptive computing strategies. Building upon the AI Flow framework, we introduce Ruyi2 as an evolution of our adaptive model series designed for efficient variable-depth computation. While early-exit architectures offer a viable efficiency-performance balance, the Ruyi model and existing methods often struggle with optimization complexity and compatibility with large-scale distributed training. To bridge this gap, Ruyi2 introduces a stable "Familial Model" based on Megatron-LM. By using 3D parallel training, it achieves a 2-3 times speedup over Ruyi, while performing comparably to same-sized Qwen3 models. These results confirm that family-based parameter sharing is a highly effective strategy, establishing a new "Train Once, Deploy Many" paradigm and providing a key reference for balancing architectural efficiency with high-performance capabilities.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 27 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

24 of 27 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  7. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  8. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  9. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  10. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  11. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  12. 2022Confident Adaptive Language Modelingcited by 297
12 more
  1. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 246
  2. 2019Depth-Adaptive Transformercited by 227
  3. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  4. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  5. 2024LLaMA Pro: Progressive LLaMA with Block Expansioncited by 99
  6. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  7. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generationcited by 64
  8. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 63
  9. 2024AI Flow at the Network Edgecited by 43
  10. 2025Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inferencecited by 18
  11. 2025HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Servingcited by 8
  12. 2025Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Ruyi2.5 Technical Reportcited by 0
  2. 2026Silicon Bureaucracy and AI Test-Oriented Education: Contamination Sensitivity and Score Confidence in LLM Benchmarkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023YAYI 2: Multilingual Open-Source Large Language Models
  2. 2026AE-LLM: Adaptive Efficiency Optimization for Large Language Models
  3. 2026Parallelization Strategies for Dense LLM Deployment: Navigating Through Application-Specific Tradeoffs and Bottlenecks
  4. 2026UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
  5. 2024Search for Efficient Large Language Models
  6. 2025Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
  7. 2025What Affects the Effective Depth of Large Language Models?
  8. 2025Accelerating Large Language Model Inference via Early-Exiting Algorithms