mlsanityData to 14 Oct 2026

Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models

arXiv 2502.18505 ↗First posted 21 Feb 2025cs.SE cs.AI cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Despite increasing discussions on open-source Artificial Intelligence (AI), existing research lacks a discussion on the transparency and accessibility of state-of-the-art (SoTA) Large Language Models (LLMs). The Open Source Initiative (OSI) has recently released its first formal definition of open-source software. This definition, when combined with standard dictionary definitions and the sparse published literature, provide an initial framework to support broader accessibility to AI models such as LLMs, but more work is essential to capture the unique dynamics of openness in AI. In addition, concerns about open-washing, where models claim openness but lack full transparency, has been raised, which limits the reproducibility, bias mitigation, and domain adaptation of these models. In this context, our study critically analyzes SoTA LLMs from the last five years, including ChatGPT, DeepSeek, LLaMA, and others, to assess their adherence to transparency standards and the implications of partial openness. Specifically, we examine transparency and accessibility from two perspectives: open-source vs. open-weight models. Our findings reveal that while some models are labeled as open-source, this does not necessarily mean they are fully open-sourced. Even in the best cases, open-source models often do not report model training data, and code as well as key metrics, such as weight accessibility, and carbon emissions. To the best of our knowledge, this is the first study that systematically examines the transparency and accessibility of over 100 different SoTA LLMs through the dual lens of open-source and open-weight models. The findings open avenues for further research and call for responsible and sustainable AI practices to ensure greater transparency, accountability, and ethical deployment of these models.(DeepSeek transparency, ChatGPT accessibility, open source, DeepSeek open source)

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 45 of its 94 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

45 of 94 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  10. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  11. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
33 more
  1. 2020Longformer: The Long-Document Transformercited by 3,413
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  11. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  13. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  14. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  15. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  16. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  17. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  18. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  19. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  20. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  21. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  22. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  23. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  24. 2023Llemma: An Open Language Model For Mathematicscited by 388
  25. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
  26. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  27. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  28. 2023Large Language Models in Finance: A Surveycited by 226
  29. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  30. 2019Explainable Machine Learning in Deploymentcited by 211
  31. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  32. 2020Behavioral Use Licensing for Responsible AIcited by 46
  33. 2023Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuningcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 1.

8 papers.