# The 2024 Foundation Model Transparency Index

arXiv: https://arxiv.org/abs/2407.12929 · HTML: https://mlsanity.com/paper/2407.12929/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 80 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 30 of 34)

- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [RewardBench: Evaluating Reward Models for Language Modeling](https://mlsanity.com/paper/2403.13787/) (2024)
- [Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research](https://mlsanity.com/paper/2402.00159/) (2024)
- [MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training](https://mlsanity.com/paper/2403.09611/) (2024)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [What Does it Mean for a Language Model to Preserve Privacy?](https://mlsanity.com/paper/2202.05520/) (2022)
- [A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity](https://mlsanity.com/paper/2305.13169/) (2023)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Principles alone cannot guarantee ethical AI](https://mlsanity.com/paper/1906.06668/) (2019)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI](https://mlsanity.com/paper/2310.16787/) (2023)
- [Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models](https://mlsanity.com/paper/2404.12387/) (2024)
- [The Foundation Model Transparency Index](https://mlsanity.com/paper/2310.12941/) (2023)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models](https://mlsanity.com/paper/2405.05374/) (2024)
- [Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus](https://mlsanity.com/paper/2105.05241/) (2021)
- [A Safe Harbor for AI Evaluation and Red Teaming](https://mlsanity.com/paper/2403.04893/) (2024)
- [Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning](https://mlsanity.com/paper/2302.08476/) (2023)
- [Ecosystem Graphs: The Social Footprint of Foundation Models](https://mlsanity.com/paper/2303.15772/) (2023)

## Built on by (most cited first, 19 of 19)

- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Jamba-1.5: Hybrid Transformer-Mamba Models at Scale](https://mlsanity.com/paper/2408.12570/) (2024)
- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://mlsanity.com/paper/2602.17753/) (2026)
- [The 2025 Foundation Model Transparency Index](https://mlsanity.com/paper/2512.10169/) (2025)
- [User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies](https://mlsanity.com/paper/2509.05382/) (2025)
- [The Role of Governments in Increasing Interconnected Post-Deployment Monitoring of AI](https://mlsanity.com/paper/2410.04931/) (2024)
- [Toxicity of the Commons: Curating Open-Source Pre-Training Data](https://mlsanity.com/paper/2410.22587/) (2024)
- [The KL3M Data Project: Copyright-Clean Training Resources for Large Language Models](https://mlsanity.com/paper/2504.07854/) (2025)
- [The Economics of AI Foundation Models: Openness, Competition, and Governance](https://mlsanity.com/paper/2510.15200/) (2025)
- [Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations](https://mlsanity.com/paper/2511.05613/) (2025)
- [Multi-agent Self-triage System with Medical Flowcharts](https://mlsanity.com/paper/2511.12439/) (2025)
- [The ATOM Report: Measuring the Open Language Model Ecosystem](https://mlsanity.com/paper/2604.07190/) (2026)
- [Generative AI in Medicine](https://mlsanity.com/paper/2412.10337/) (2024)
- [eDIF: A European Deep Inference Fabric for Remote Interpretability of LLM](https://mlsanity.com/paper/2508.10553/) (2025)
- [Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmap](https://mlsanity.com/paper/2508.18646/) (2025)
- [Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs](https://mlsanity.com/paper/2606.12385/) (2026)
- [Position: AI for Science Should Treat Measurement-to-Dataset Pipelines as Inference Components](https://mlsanity.com/paper/2605.24558/) (2026)
- [Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing](https://mlsanity.com/paper/2606.00686/) (2026)
- [Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers](https://mlsanity.com/paper/2609.08789/) (2026)

## Abstract (arXiv)

Foundation models are increasingly consequential yet extremely opaque. To characterize the status quo, the Foundation Model Transparency Index (FMTI) was launched in October 2023 to measure the transparency of leading foundation model developers. FMTI 2023 assessed 10 major foundation model developers (e.g. OpenAI, Google) on 100 transparency indicators (e.g. does the developer disclose the wages it pays for data labor?). At the time, developers publicly disclosed very limited information with the average score being 37 out of 100. To understand how the status quo has changed, we conduct a follow-up study after 6 months: we score 14 developers against the same 100 indicators. While in FMTI 2023 we searched for publicly available information, in FMTI 2024 developers submit reports on the 100 transparency indicators, potentially including information that was not previously public. We find that developers now score 58 out of 100 on average, a 21 point improvement over FMTI 2023. Much of this increase is driven by developers disclosing information during the FMTI 2024 process: on average, developers disclosed information related to 16.6 indicators that was not previously public. We observe regions of sustained (i.e. across 2023 and 2024) and systemic (i.e. across most or all developers) opacity such as on copyright status, data access, data labor, and downstream impact. We publish transparency reports for each developer that consolidate information disclosures: these reports are based on the information disclosed to us via developers. Our findings demonstrate that transparency can be improved in this nascent ecosystem, the Foundation Model Transparency Index likely contributes to these improvements, and policymakers should consider interventions in areas where transparency has not improved.