mlsanityData to 7 Oct 2026

The Llama 3 Herd of Models

arXiv 2407.21783 ↗First posted 31 Jul 2024cs.AI cs.CL cs.CVcited by 14,192

Not on the map. Why?

Cited by 2,589 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Key facts. Cited by 14,192 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Jul 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1902; 2025: 7226; 2026: 5064.

14,192 papers.

Sort:

By area:

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  5. 2025Gemma 3 Technical Reportcited by 1,686
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  11. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
38 more
  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  2. 2025Large Language Diffusion Modelscited by 872
  3. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  4. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  5. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  6. 2025Qwen2.5-Omni Technical Reportcited by 704
  7. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  8. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  9. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  10. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  11. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  12. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  13. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  14. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  15. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  16. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  17. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  18. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  19. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  20. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
  21. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  22. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  23. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  24. 2025Muon is Scalable for LLM Trainingcited by 381
  25. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  26. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  27. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  28. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  29. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
  30. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  31. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  32. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  33. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  34. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  35. 2024Alignment faking in large language modelscited by 316
  36. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  37. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  38. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306

50 most cited of 14,192 All 14,192 (JSON)