mlsanityData to 14 Oct 2026

The Llama 4 Herd: Architecture, Training, Evaluation, and Deployment Notes

arXiv 2601.11659 ↗First posted 15 Jan 2026cs.SE cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

This document consolidates publicly reported technical details about Metas Llama 4 model family. It summarizes (i) released variants (Scout and Maverick) and the broader herd context including the previewed Behemoth teacher model, (ii) architectural characteristics beyond a high-level MoE description covering routed/shared-expert structure, early-fusion multimodality, and long-context design elements reported for Scout (iRoPE and length generalization strategies), (iii) training disclosures spanning pre-training, mid-training for long-context extension, and post-training methodology (lightweight SFT, online RL, and lightweight DPO) as described in release materials, (iv) developer-reported benchmark results for both base and instruction-tuned checkpoints, and (v) practical deployment constraints observed across major serving environments, including provider-specific context limits and quantization packaging. The manuscript also summarizes licensing obligations relevant to redistribution and derivative naming, and reviews publicly described safeguards and evaluation practices. The goal is to provide a compact technical reference for researchers and practitioners who need precise, source-backed facts about Llama 4.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 0 of its 11 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

0 of 11 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 30.

30 papers.

Sort:

By area:

  1. 2026LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routerscited by 4
  2. 2026Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoningcited by 3
  3. 2026Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligencecited by 3
  4. 2026An Empirical Study on Noisy Data and LLM Pretraining Loss Divergencecited by 2
  5. 2026Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLMcited by 2
  6. 2026ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Modelscited by 2
  7. 2026Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generationcited by 1
  8. 2026Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognitioncited by 1
  9. 2026WARBENCH: A Comprehensive Benchmark for Evaluating LLMs in Military Decision-Makingcited by 1
  10. 2026SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identificationcited by 1
  11. 2026Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansioncited by 1
  12. 2026Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoningcited by 1
18 more
  1. 2026Maximum Likelihood Estimation for Entity Ranking under Iterative Synthetic Data Augmentationcited by 1
  2. 2026MPCEval: A Benchmark for Multi-Party Conversation Generationcited by 0
  3. 2026Domain Adapted Large Language Models for Additive Manufacturingcited by 0
  4. 2026GazeVaLM: A Multi-Observer Eye-Tracking Benchmark for Evaluating Clinical Realism in AI-Generated X-Rayscited by 0
  5. 2026TIDE-Bench: Task-Aware and Diagnostic Evaluation of Tool-Integrated Reasoningcited by 0
  6. 2026Tracing the ongoing emergence of human-like reasoning in Large Language Modelscited by 0
  7. 2026OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfieldscited by 0
  8. 2026Saliency-Aware Model Mergingcited by 0
  9. 2026EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summariescited by 0
  10. 2026NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Researchcited by 0
  11. 2026Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devicescited by 0
  12. 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinationscited by 0
  13. 2026Physico-Geospatial Grounded Scene Interpretation for Mobile Roboticscited by 0
  14. 2026Syndrome, Synergy, and Safety: Structured Reasoning and Knowledge-Driven Alignment for TCM Prescription Generationcited by 0
  15. 2026One Model Is Not a Crowd: Multi-LLM and Aspect-Conditioned Diverse Comment Generationcited by 0
  16. 2026PowerBench: A Benchmark for Agentic Retrieval and Reasoning in Power Systemscited by 0
  17. 2026AraDynFact: Dynamic Evaluation of Factual Knowledge in Arabiccited by 0
  18. 2026Correct Verdicts, Flawed Reasoning: Structured Auditing of LLM-based Vulnerability Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Llama-Nemotron: Efficient Reasoning Models
  2. 2026Benchmarking LLAMA Model Security Against OWASP Top 10 For LLM Applications
  3. 2023LLaMA: Open and Efficient Foundation Language Models
  4. 2026MinT: Managed Infrastructure for Training and Serving Millions of LLMs
  5. 2026Herding CATs: ALARA for Agent Harness Engineering in Portable Composable Multi-Agent Teams
  6. 2025Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey
  7. 2023Code Llama: Open Foundation Models for Code
  8. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions