mlsanityData to 7 Oct 2026

Llama 2: Open Foundation and Fine-Tuned Chat Models

arXiv 2307.09288 ↗First posted 18 Jul 2023cs.CL cs.AIcited by 12,865

Not on the map. Why?

Cited by 1,366 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.

Key facts. Cited by 12,865 papers in the map (data through 7 Oct 2026). Builds on 98 of its 131 references that are in the map. First posted on arXiv 18 Jul 2023.

Builds on

98 of 131 references.

References · 98

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2020Scaling Laws for Neural Language Modelscited by 6,411
  11. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  12. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
38 more
  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  13. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  14. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  15. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  16. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  17. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  18. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  19. 2020Learning to summarize from human feedbackcited by 2,562
  20. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  21. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  22. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  23. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  24. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  25. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  26. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  27. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  28. 2020GLU Variants Improve Transformercited by 1,724
  29. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  30. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  31. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  32. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  33. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  34. 2018Model Cards for Model Reportingcited by 1,068
  35. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  36. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  37. 2021Ethical and social risks of harm from Language Modelscited by 957
  38. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1431; 2024: 6297; 2025: 3608; 2026: 1529.

12,865 papers.

Sort:

By area:

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2023Mistral 7Bcited by 3,065
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  10. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  12. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
38 more
  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  9. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  10. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  11. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  13. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  14. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  15. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  16. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  17. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  18. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  19. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  20. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  21. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  22. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  23. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  24. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  25. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  26. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  27. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  28. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  29. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  30. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  31. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  32. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  33. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  34. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  35. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  36. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  37. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  38. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893

50 most cited of 12,865 All 12,865 (JSON)