mlsanityData to 14 Oct 2026

The 2025 Foundation Model Transparency Index

arXiv 2512.10169 ↗First posted 11 Dec 2025cs.AI cs.CY cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Foundation model developers are among the world's most important companies. As these companies become increasingly consequential, how do their transparency practices evolve? The 2025 Foundation Model Transparency Index is the third edition of an annual effort to characterize and quantify the transparency of foundation model developers. The 2025 FMTI introduces new indicators related to data acquisition, usage data, and monitoring and evaluates companies like Alibaba, DeepSeek, and xAI for the first time. The 2024 FMTI reported that transparency was improving, but the 2025 FMTI finds this progress has deteriorated: the average score out of 100 fell from 58 in 2024 to 40 in 2025. Companies are most opaque about their training data and training compute as well as the post-deployment usage and impact of their flagship models. In spite of this general trend, IBM stands out as a positive outlier, scoring 95, in contrast to the lowest scorers, xAI and Midjourney, at just 14. The five members of the Frontier Model Forum we score end up in the middle of the Index: we posit that these companies avoid reputational harms from low scores but lack incentives to be transparency leaders. As policymakers around the world increasingly mandate certain types of transparency, this work reveals the current state of transparency for foundation model developers, how it may change given newly enacted policy, and where more aggressive policy interventions are necessary to address critical information deficits.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 48 of its 98 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

48 of 98 references.

References · 48

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  4. 2018Model Cards for Model Reportingcited by 1,080
  5. 2021Ethical and social risks of harm from Language Modelscited by 966
  6. 2019Green AIcited by 627
  7. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  8. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  9. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  10. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
  11. 2023Scaling Data-Constrained Language Modelscited by 403
  12. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
36 more
  1. 2023How is ChatGPT's behavior changing over time?cited by 270
  2. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  3. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 182
  4. 2020Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimscited by 165
  5. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  6. 2018Troubling Trends in Machine Learning Scholarshipcited by 108
  7. 2025A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)cited by 101
  8. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 88
  9. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  10. 2023The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AIcited by 73
  11. 2023The Foundation Model Transparency Indexcited by 66
  12. 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
  13. 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
  14. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
  15. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  16. 2022Structured access: an emerging paradigm for safe AI deploymentcited by 41
  17. 2023Understanding accountability in algorithmic supply chainscited by 40
  18. 2023Ecosystem Graphs: The Social Footprint of Foundation Modelscited by 32
  19. 2025ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilingualitycited by 22
  20. 2024The 2024 Foundation Model Transparency Indexcited by 19
  21. 2024Language model developers should report train-test overlapcited by 16
  22. 2023Market Concentration Implications of Foundation Modelscited by 15
  23. 2024Foundation Model Transparency Reportscited by 14
  24. 2024Responsible Reporting for Frontier AI Developmentcited by 10
  25. 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reportscited by 10
  26. 2025User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policiescited by 9
  27. 2025Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipelinecited by 8
  28. 2025In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AIcited by 7
  29. 2023I'm Afraid I Can't Do That: Predicting Prompt Refusal in Black-Box Generative Language Modelscited by 6
  30. 2025Designing Incident Reporting Systems for Harms from General-Purpose AIcited by 6
  31. 2020Trustworthy AI Inference Systems: An Industry Research Viewcited by 4
  32. 2025Policy Frameworks for Transparent Chain-of-Thought Reasoning in Large Language Modelscited by 4
  33. 2025Practical Principles for AI Cost and Compute Accountingcited by 3
  34. 2023Thinking Upstream: Ethics and Policy Opportunities in AI Supply Chainscited by 2
  35. 2024Effective Mitigations for Systemic Risks from General-Purpose AIcited by 2
  36. 2025Do AI Companies Make Good on Voluntary Commitments to the White House?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 30
  2. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  3. 2026The ATOM Report: Measuring the Open Language Model Ecosystemcited by 2
  4. 2026M"OVE: A Holistic LLM Benchmark for the German Public Sectorcited by 1
  5. 2026Selecting Language Models for Social Science: Start Small, Start Open, and Validatecited by 0
  6. 2026NeurIPS Should Require Reproducibility Standards for Frontier AI Safety Claimscited by 0
  7. 2026Short-Term Gain, Long-Term Fragility: AI Labor Substitution and the Erosion of Sustainable Capabilitycited by 0
  8. 2026Neuro-Symbolic Software Verification: Hyper-charging Local Language Models with Symbolic Reasoning at Scalecited by 0
  9. 2026EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failurescited by 0
  10. 2026Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developerscited by 0
  11. 2026Jailbreaks for Black-Box Uncertainty Quantification in Large Reasoning Modelscited by 0