mlsanityData to 14 Oct 2026

Baichuan 2: Open Large-scale Language Models

arXiv 2309.10305 ↗First posted 19 Sep 2023cs.CLcited by 670

Not on the map. Why?

Cited by 56 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering. However, most powerful LLMs are closed-source or limited in their capability for languages other than English. In this technical report, we present Baichuan 2, a series of large-scale multilingual language models containing 7 billion and 13 billion parameters, trained from scratch, on 2.6 trillion tokens. Baichuan 2 matches or outperforms other open-source models of similar size on public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval. Furthermore, Baichuan 2 excels in vertical domains such as medicine and law. We will release all pre-training model checkpoints to benefit the research community in better understanding the training dynamics of Baichuan 2.

Key facts. Cited by 670 papers in the map (data through 14 Oct 2026). Builds on 61 of its 83 references that are in the map. First posted on arXiv 19 Sep 2023.

Builds on

61 of 83 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2017Proximal Policy Optimization Algorithmscited by 15,215
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  11. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  12. 2020Scaling Laws for Neural Language Modelscited by 6,586
38 more
  1. 2016Layer Normalizationcited by 5,669
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  13. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  14. 2018Deep contextualized word representationscited by 2,290
  15. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  16. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  17. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  18. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  19. 2020GLU Variants Improve Transformercited by 1,804
  20. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  21. 2019Root Mean Square Layer Normalizationcited by 1,480
  22. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  23. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  24. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  25. 2023PaLM 2 Technical Reportcited by 1,162
  26. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  27. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  28. 2020On Layer Normalization in the Transformer Architecturecited by 922
  29. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  30. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  31. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  32. 2022Galactica: A Large Language Model for Sciencecited by 744
  33. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  34. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  35. 2016Language Modeling with Gated Convolutional Networkscited by 702
  36. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  37. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  38. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 91; 2024: 412; 2025: 130; 2026: 37.

670 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,344
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  6. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  7. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  8. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  9. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  10. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  11. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 301
  12. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
38 more
  1. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  2. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
  3. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  4. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 254
  5. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  6. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  7. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  8. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
  9. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  10. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  11. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  12. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 188
  13. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 186
  14. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  15. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 174
  16. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
  17. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
  18. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
  19. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  20. 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 144
  21. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 141
  22. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 134
  23. 2024SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Modelscited by 131
  24. 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 126
  25. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  26. 2024Why Transformers Need Adam: A Hessian Perspectivecited by 124
  27. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
  28. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  29. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  30. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 111
  31. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  32. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 107
  33. 2024Benchmarking Benchmark Leakage in Large Language Modelscited by 106
  34. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 105
  35. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 102
  36. 2023Learning From Mistakes Makes LLM Better Reasonercited by 101
  37. 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulatorcited by 101
  38. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Headscited by 100

50 most cited of 670 All 670 (JSON)