# a survey on GPT-3

arXiv: https://arxiv.org/abs/2212.00857 · HTML: https://mlsanity.com/paper/2212.00857/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 40 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text](https://mlsanity.com/paper/2107.01294/) (2021)
- [Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization](https://mlsanity.com/paper/2110.07356/) (2021)
- [GPT3-to-plan: Extracting plans from text using GPT-3](https://mlsanity.com/paper/2106.07131/) (2021)
- [Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management](https://mlsanity.com/paper/2108.01764/) (2021)
- [CGEMs: A Metric Model for Automatic Code Generation using GPT-3](https://mlsanity.com/paper/2108.10168/) (2021)

## Built on by (most cited first, 9 of 9)

- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Can ChatGPT Assess Human Personalities? A General Evaluation Framework](https://mlsanity.com/paper/2303.01248/) (2023)
- [Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers](https://mlsanity.com/paper/2311.10242/) (2023)
- [BIM-GPT: a Prompt-Based Virtual Assistant Framework for BIM Information Retrieval](https://mlsanity.com/paper/2304.09333/) (2023)
- [Hallucination-minimized Data-to-answer Framework for Financial Decision-makers](https://mlsanity.com/paper/2311.07592/) (2023)
- [Ta'keed: The First Generative Fact-Checking System for Arabic Claims](https://mlsanity.com/paper/2401.14067/) (2024)
- [Implicit Causality-biases in humans and LLMs as a tool for benchmarking LLM discourse capabilities](https://mlsanity.com/paper/2501.12980/) (2025)
- [How LLMs are Shaping the Future of Virtual Reality](https://mlsanity.com/paper/2508.00737/) (2025)
- [NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference](https://mlsanity.com/paper/2602.02988/) (2026)

## Abstract (arXiv)

This paper provides an introductory survey to GPT-3. We cover some of the historical development behind this technology, some of the key features of GPT-3, and discuss the machine learning model and the datasets used. We survey both academic and commercial efforts applying GPT-3 in diverse domains such as developing conversational AI chatbots, software development, creative work, domain knowledge, and business productivity. We discuss some of the challenges that GPT-3 faces such as the problems of training complexity, bias, and hallucination/incorrect answers. We also discuss the future research opportunities in this area.