# From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequences

arXiv: https://arxiv.org/abs/2604.10332 · HTML: https://mlsanity.com/paper/2604.10332/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 67 references that are in the map.
- First posted on arXiv 11 Apr 2026.

## Builds on (most cited first, 30 of 35)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination](https://mlsanity.com/paper/2507.10532/) (2025)
- [Continual Instruction Tuning for Large Multimodal Models](https://mlsanity.com/paper/2311.16206/) (2023)
- [Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing](https://mlsanity.com/paper/2508.12631/) (2025)
- [Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency](https://mlsanity.com/paper/2407.09519/) (2024)
- [Bias Unveiled: Investigating Social Bias in LLM-Generated Code](https://mlsanity.com/paper/2411.10351/) (2024)
- [Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression](https://mlsanity.com/paper/2508.05337/) (2025)
- [Graph Generative Pre-trained Transformer](https://mlsanity.com/paper/2501.01073/) (2025)
- [LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users](https://mlsanity.com/paper/2406.17737/) (2024)
- [A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgment](https://mlsanity.com/paper/2504.12408/) (2025)
- [Evaluating Long-Context Reasoning in LLM-Based WebAgents](https://mlsanity.com/paper/2512.04307/) (2025)
- [GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries](https://mlsanity.com/paper/2508.00033/) (2025)
- [Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)](https://mlsanity.com/paper/2510.04950/) (2025)
- [HumanEval on Latest GPT Models -- 2024](https://mlsanity.com/paper/2402.14852/) (2024)
- [No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biases](https://mlsanity.com/paper/2511.18635/) (2025)
- [Capabilities of GPT-5 across critical domains: Is it the next breakthrough?](https://mlsanity.com/paper/2508.19259/) (2025)
- [Institutional AI: A Governance Framework for Distributional AGI Safety](https://mlsanity.com/paper/2601.10599/) (2026)
- [When Benchmarks Leak: Inference-Time Decontamination for LLMs](https://mlsanity.com/paper/2601.19334/) (2026)
- [The Path Ahead for Agentic AI: Challenges and Opportunities](https://mlsanity.com/paper/2601.02749/) (2026)
- [HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering](https://mlsanity.com/paper/2603.18558/) (2026)

## Built on by (most cited first, 3 of 3)

- [Benchmarking LLM Judges for Voice-Agent Evaluation: Reliability, Calibration, and Human Oversight](https://mlsanity.com/paper/2608.24314/) (2026)
- [VDGR-RAG: Vectors, Directories, Graphs, and Reflection Are All You Need for Unified Reasoning over Hierarchical Enterprise Knowledge](https://mlsanity.com/paper/2608.07994/) (2026)
- [CRISS: A Retrieval-Augmented AI Chatbot for Assisting Cancer Registrars](https://mlsanity.com/paper/2609.29075/) (2026)

## Abstract (arXiv)

We present the progress of the GPT family from GPT-3 through GPT-3.5, GPT-4, GPT-4 Turbo, GPT-4o, GPT-4.1, and the GPT-5 family. Our work is comparative rather than merely historical. We investigates how the family evolved in technical framing, user interaction, modality, deployment architecture, and governance viewpoint. The work focuses on five recurring themes: technical progression, capability changes, deployment shifts, persistent limitations, and downstream consequences. In term of research design, we consider official technical reports, system cards, API and model documentation, product announcements, release notes, and peer-reviewed secondary studies. A primary assertion is that later GPT generations should not be interpreted only as larger or more accurate language models. Instead, the family evolves from a scaled few-shot text predictor into a set of aligned, multimodal, tool-oriented, long-context, and increasingly workflow-integrated systems. This development complicates simple model-to-model comparison because product routing, tool access, safety tuning, and interface design become part of the effective system. Across generations, several limitations remain unchanged: hallucination, prompt sensitivity, benchmark fragility, uneven behavior across domains and populations, and incomplete public transparency about architecture and training. However, the family has evolved software development, educational practice, information work, interface design, and discussions of frontier-model governance. We infer that the transition from GPT-3 to GPT-5 is best understood not only as an improvement in model capability, but also as a broader reformulation of what a deployable AI system is, how it is evaluated, and where responsibility should be located when such systems are used at scale.