mlsanityData to 7 Oct 2026

Qwen2 Technical Report

arXiv 2407.10671 ↗First posted 15 Jul 2024cs.CL cs.AIcited by 2,076

Not on the map. Why?

Cited by 524 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models. We release a comprehensive suite of foundational and instruction-tuned language models, encompassing a parameter range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts model. Qwen2 surpasses most prior open-weight models, including its predecessor Qwen1.5, and exhibits competitive performance relative to proprietary models across diverse benchmarks on language understanding, generation, multilingual proficiency, coding, mathematics, and reasoning. The flagship model, Qwen2-72B, showcases remarkable performance: 84.2 on MMLU, 37.9 on GPQA, 64.6 on HumanEval, 89.5 on GSM8K, and 82.4 on BBH as a base language model. The instruction-tuned variant, Qwen2-72B-Instruct, attains 9.1 on MT-Bench, 48.1 on Arena-Hard, and 35.7 on LiveCodeBench. Moreover, Qwen2 demonstrates robust multilingual capabilities, proficient in approximately 30 languages, spanning English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, Vietnamese, and more, underscoring its versatility and global reach. To foster community innovation and accessibility, we have made the Qwen2 model weights openly available on Hugging Face and ModelScope, and the supplementary materials including example code on GitHub. These platforms also include resources for quantization, fine-tuning, and deployment, facilitating a wide range of applications and research endeavors.

Key facts. Cited by 2,076 papers in the map (data through 7 Oct 2026). Builds on 60 of its 77 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

60 of 77 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  12. 2021Program Synthesis with Large Language Modelscited by 3,509
38 more
  1. 2023Qwen Technical Reportcited by 3,273
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  6. 2024Mixtral of Expertscited by 1,785
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  11. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  13. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  14. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  15. 2024Yi: Open Foundation Models by 01.AIcited by 706
  16. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  17. 2016Language Modeling with Gated Convolutional Networkscited by 688
  18. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  19. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  20. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  21. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  22. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  23. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  24. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
  25. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 390
  26. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  27. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  28. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  29. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  30. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  31. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  32. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  33. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 207
  34. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  35. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  36. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 108
  37. 2024Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignmentcited by 104
  38. 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 80

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 529; 2025: 980; 2026: 567.

2,076 papers.

Sort:

By area:

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  6. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  7. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  8. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  9. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  10. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
  11. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  12. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
38 more
  1. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  2. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  3. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  4. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  5. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  6. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  7. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  8. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
  9. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  10. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 255
  11. 2025TTRL: Test-Time Reinforcement Learningcited by 253
  12. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  13. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  14. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
  15. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  16. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 204
  17. 2025Generalizing Verifiable Instruction Followingcited by 178
  18. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  19. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  20. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 157
  21. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  22. 2024Multimodal Whole Slide Foundation Model for Pathologycited by 156
  23. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
  24. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 153
  25. 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 153
  26. 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 151
  27. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 150
  28. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  29. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakagecited by 146
  30. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  31. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  32. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  33. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  34. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
  35. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
  36. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 132
  37. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 132
  38. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132

50 most cited of 2,076 All 2,076 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Qwen3-VL Technical Report
  2. 2024Qwen2-Audio Technical Report
  3. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  4. 2025Multilingual Language Model Pretraining using Machine-translated Data
  5. 2023YAYI 2: Multilingual Open-Source Large Language Models
  6. 2025LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning
  7. 2024What Drives Performance in Multilingual Language Models?
  8. 2024BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment