mlsanityData to 7 Oct 2026

Solar Open Technical Report

arXiv 2601.07022 ↗First posted 11 Jan 2026cs.CLcited by 2

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

We introduce Solar Open, a 102B-parameter bilingual Mixture-of-Experts language model for underserved languages. Solar Open demonstrates a systematic methodology for building competitive LLMs by addressing three interconnected challenges. First, to train effectively despite data scarcity for underserved languages, we synthesize 4.5T tokens of high-quality, domain-specific, and RL-oriented data. Second, we coordinate this data through a progressive curriculum jointly optimizing composition, quality thresholds, and domain coverage across 20 trillion tokens. Third, to enable reasoning capabilities through scalable RL, we apply our proposed framework SnapPO for efficient optimization. Across benchmarks in English and Korean, Solar Open achieves competitive performance, demonstrating the effectiveness of this methodology for underserved language AI development.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 54 of its 67 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

54 of 67 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Qwen3 Technical Report14 May
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer23 Jan

Other references · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  10. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  11. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
38 more
  1. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  2. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  3. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  4. 2025Group Sequence Policy Optimizationcited by 788
  5. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  6. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  7. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  8. 2025Humanity's Last Examcited by 467
  9. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  10. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  11. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 313
  12. 20242 OLMo 2 Furiouscited by 302
  13. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  14. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  15. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  16. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  17. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
  18. 2025Skywork Open Reasoner 1 Technical Reportcited by 161
  19. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  20. 2025FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Languagecited by 134
  21. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  22. 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 95
  23. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  24. 2024TorchTitan: One-stop PyTorch native solution for production ready LLM pre-trainingcited by 80
  25. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 65
  26. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 64
  27. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 63
  28. 2025Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-trainingcited by 62
  29. 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
  30. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 41
  31. 2022KOBEST: Korean Balanced Evaluation of Significant Taskscited by 37
  32. 2024LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMscited by 31
  33. 2024KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean Healthcare Professional Licensing Examinationscited by 22
  34. 2025KoBALT: Korean Benchmark For Advanced Linguistic Taskscited by 12
  35. 2025Atomic Skills are the Prerequisite: When Reinforcement Learning Synthesizes Compositional Reasoning, and When It Only Amplifiescited by 8
  36. 2024Open Ko-LLM Leaderboard2: Bridging Foundational and Practical Evaluation for Korean LLMscited by 6
  37. 2025Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoningcited by 6
  38. 2024Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Modelscited by 4

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Modelscited by 5
  2. 2026Rufus-Air: An Open LLM Post-Training Recipecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Solar Open 2 Technical Report
  2. 2025FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
  3. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
  4. 2026Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
  5. 2025Mixture-of-Experts Can Surpass Dense LLMs Under Strictly Equal Resource
  6. 2024Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
  7. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
  8. 2025Democratizing LLM Efficiency: From Hyperscale Optimizations to Universal Deployability