mlsanityData to 7 Oct 2026

Phi-4 Technical Report

arXiv 2412.08905 ↗First posted 12 Dec 2024cs.CL cs.AIcited by 675

Not on the map. Why?

Cited by 190 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present phi-4, a 14-billion parameter language model developed with a training recipe that is centrally focused on data quality. Unlike most language models, where pre-training is based primarily on organic data sources such as web content or code, phi-4 strategically incorporates synthetic data throughout the training process. While previous models in the Phi family largely distill the capabilities of a teacher model (specifically GPT-4), phi-4 substantially surpasses its teacher model on STEM-focused QA capabilities, giving evidence that our data-generation and post-training techniques go beyond distillation. Despite minimal changes to the phi-3 architecture, phi-4 achieves strong performance relative to its size -- especially on reasoning-focused benchmarks -- due to improved data, training curriculum, and innovations in the post-training scheme.

Key facts. Cited by 675 papers in the map (data through 7 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

19 of 36 references.

References · 19

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  10. 2023Textbooks Are All You Needcited by 561
  11. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  12. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
7 more
  1. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  2. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  3. 2024Measuring short-form factuality in large language modelscited by 345
  4. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  5. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 80
  6. 2024Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cyclecited by 32
  7. 2023A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applicationscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 391; 2026: 280.

675 papers.

Sort:

By area:

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  3. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  4. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  5. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  6. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  7. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  8. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 127
  9. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 126
  10. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  11. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 110
  12. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
38 more
  1. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 84
  2. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  3. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 73
  4. 2025Distillation Scaling Lawscited by 59
  5. 2025BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoningcited by 57
  6. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
  7. 2025Training a Scientific Reasoning Model for Chemistrycited by 54
  8. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  9. 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 53
  10. 2025LEXam: Benchmarking Legal Reasoning on 340 Law Examscited by 52
  11. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 51
  12. 2025Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operationscited by 46
  13. 2025Scaling Laws of Synthetic Data for Language Modelscited by 44
  14. 2025Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modelingcited by 44
  15. 2025Reasoning to Learn from Latent Thoughtscited by 43
  16. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Modelscited by 43
  17. 2025DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Searchcited by 41
  18. 2025Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Datasetcited by 34
  19. 2025Idiosyncrasies in Large Language Modelscited by 32
  20. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Taskscited by 32
  21. 2025Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layerscited by 32
  22. 2025Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Modelscited by 31
  23. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  24. 2025CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning taskscited by 30
  25. 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 30
  26. 2025Thought Communication in Multiagent Collaborationcited by 28
  27. 2025LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuningcited by 27
  28. 2025AlphaOne: Reasoning Models Thinking Slow and Fast at Test Timecited by 27
  29. 2025EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modescited by 26
  30. 2024Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Contextcited by 25
  31. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 25
  32. 2026BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoningcited by 25
  33. 2025Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Insteadcited by 24
  34. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfallscited by 24
  35. 2025Wait, Wait, Wait... Why Do Reasoning Models Loop?cited by 24
  36. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  37. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 23
  38. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environmentscited by 23

50 most cited of 675 All 675 (JSON)