mlsanityData to 7 Oct 2026

Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Models

arXiv 2308.14149 ↗First posted 27 Aug 2023cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Generative pre-trained transformer (GPT) models have revolutionized the field of natural language processing (NLP) with remarkable performance in various tasks and also extend their power to multimodal domains. Despite their success, large GPT models like GPT-4 face inherent limitations such as considerable size, high computational requirements, complex deployment processes, and closed development loops. These constraints restrict their widespread adoption and raise concerns regarding their responsible development and usage. The need for user-friendly, relatively small, and open-sourced alternative GPT models arises from the desire to overcome these limitations while retaining high performance. In this survey paper, we provide an examination of alternative open-sourced models of large GPTs, focusing on user-friendly and relatively small models that facilitate easier deployment and accessibility. Through this extensive survey, we aim to equip researchers, practitioners, and enthusiasts with a thorough understanding of user-friendly and relatively small open-sourced models of large GPTs, their current state, challenges, and future research directions, inspiring the development of more efficient, accessible, and versatile GPT models that cater to the broader scientific community and advance the field of general artificial intelligence. The source contents are continuously updating in https://github.com/GPT-Alternatives/gpt_alternatives.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 122 of its 177 references that are in the map. First posted on arXiv 27 Aug 2023.

Builds on

122 of 177 references.

References · 122

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
38 more
  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  13. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  14. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  15. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  16. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  17. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  18. 2023Let's Verify Step by Stepcited by 3,598
  19. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  20. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  21. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  22. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  23. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  24. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  25. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  26. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  27. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  28. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  29. 2020Learning to summarize from human feedbackcited by 2,562
  30. 2023A Survey of Large Language Modelscited by 2,537
  31. 2015VQA: Visual Question Answeringcited by 2,489
  32. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  33. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  34. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  35. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  36. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  37. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  38. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  2. 2024GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Modelscited by 7
  3. 2025Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generationcited by 2
  4. 2025LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generationcited by 0