mlsanityData to 7 Oct 2026

Qwen Technical Report

arXiv 2309.16609 ↗First posted 28 Sep 2023cs.CLcited by 3,273

Not on the map. Why?

Cited by 626 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans. In this work, we introduce Qwen, the first installment of our large language model series. Qwen is a comprehensive language model series that encompasses distinct models with varying parameter counts. It includes Qwen, the base pretrained language models, and Qwen-Chat, the chat models finetuned with human alignment techniques. The base language models consistently demonstrate superior performance across a multitude of downstream tasks, and the chat models, particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are highly competitive. The chat models possess advanced tool-use and planning capabilities for creating agent applications, showcasing impressive performance even when compared to bigger models on complex tasks like utilizing a code interpreter. Furthermore, we have developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models, Math-Qwen-Chat, which are built upon base language models. These models demonstrate significantly improved performance in comparison with open-source models, and slightly fall behind the proprietary models.

Key facts. Cited by 3,273 papers in the map (data through 7 Oct 2026). Builds on 123 of its 164 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

123 of 164 references.

References · 123

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  12. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
38 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  12. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  13. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  14. 2017Deep reinforcement learning from human preferencescited by 3,846
  15. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  16. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  17. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  18. 2021Program Synthesis with Large Language Modelscited by 3,509
  19. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  20. 2020Longformer: The Long-Document Transformercited by 3,362
  21. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  22. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  23. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  24. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  25. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  26. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  27. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  28. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  29. 2020Learning to summarize from human feedbackcited by 2,562
  30. 2022Emergent Abilities of Large Language Modelscited by 2,483
  31. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  32. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  33. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  34. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  35. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  36. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  37. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  38. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 76; 2024: 1055; 2025: 1369; 2026: 773.

3,273 papers.

Sort:

By area:

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2024Qwen2 Technical Reportcited by 2,076
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  9. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
38 more
  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  3. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  6. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  7. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  8. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  9. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  10. 2024DeepSeek-V3 Technical Reportcited by 724
  11. 2025Qwen2.5-Omni Technical Reportcited by 704
  12. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  13. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  14. 2024A Survey on Large Language Models for Code Generationcited by 646
  15. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  16. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  17. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  18. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  19. 2024Qwen2-Audio Technical Reportcited by 543
  20. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  21. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  22. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  23. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  24. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  25. 2025Qwen3-Omni Technical Reportcited by 451
  26. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  27. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  28. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  29. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 394
  30. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  31. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  32. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  33. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 346
  34. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  35. 2024LoRA Learns Less and Forgets Lesscited by 313
  36. 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 300
  37. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 298
  38. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298

50 most cited of 3,273 All 3,273 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages
  2. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  3. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results
  4. 2023A Survey on Large Language Model based Autonomous Agents
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  7. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  8. 2024A critical review of methods and challenges in large language models