# The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence

arXiv: https://arxiv.org/abs/2403.13784 · HTML: https://mlsanity.com/paper/2403.13784/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 120 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models](https://mlsanity.com/paper/2310.02949/) (2023)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [SantaCoder: don't reach for the stars!](https://mlsanity.com/paper/2301.03988/) (2023)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence](https://mlsanity.com/paper/2402.09880/) (2024)
- [On the Societal Impact of Open Foundation Models](https://mlsanity.com/paper/2403.07918/) (2024)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators](https://mlsanity.com/paper/2307.05532/) (2023)
- [AI auditing: The Broken Bus on the Road to AI Accountability](https://mlsanity.com/paper/2401.14462/) (2024)
- [Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives](https://mlsanity.com/paper/2311.09227/) (2023)
- [Typhoon: Thai Large Language Models](https://mlsanity.com/paper/2312.13951/) (2023)
- [A Framework for Fairness: A Systematic Review of Existing Fair AI Solutions](https://mlsanity.com/paper/2112.05700/) (2021)
- [A Cost Analysis of Generative Language Models and Influence Operations](https://mlsanity.com/paper/2308.03740/) (2023)
- [EleutherAI: Going Beyond "Open Science" to "Science in the Open"](https://mlsanity.com/paper/2210.06413/) (2022)

## Built on by (most cited first, 23 of 23)

- [MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages](https://mlsanity.com/paper/2410.01036/) (2024)
- [MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application](https://mlsanity.com/paper/2506.14028/) (2025)
- [A Survey of Large Language Models for Arabic Language and its Dialects](https://mlsanity.com/paper/2410.20238/) (2024)
- [What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practices](https://mlsanity.com/paper/2504.03803/) (2025)
- [An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face](https://mlsanity.com/paper/2502.04484/) (2025)
- [7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement](https://mlsanity.com/paper/2412.06845/) (2024)
- [Bridging LLM-Generated Code and Requirements: Reverse Generation technique and SBC Metric for Developer Insights](https://mlsanity.com/paper/2502.07835/) (2025)
- [The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI](https://mlsanity.com/paper/2501.18669/) (2025)
- [Reflexive Prompt Engineering: A Framework for Responsible Prompt Engineering and Interaction Design](https://mlsanity.com/paper/2504.16204/) (2025)
- [FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian](https://mlsanity.com/paper/2505.22759/) (2025)
- [How Open Must Language Models be to Enable Reliable Scientific Inference?](https://mlsanity.com/paper/2603.26539/) (2026)
- [Beyond Release: Access Considerations for Generative AI Systems](https://mlsanity.com/paper/2502.16701/) (2025)
- [Open and Sustainable AI: challenges, opportunities and the road ahead in the life sciences (October 2025 -- Version 2)](https://mlsanity.com/paper/2505.16619/) (2025)
- [The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergence](https://mlsanity.com/paper/2505.23420/) (2025)
- [The Shift Toward Open and Reproducible AI Research](https://mlsanity.com/paper/2606.16974/) (2026)
- [Opening the Scope of Openness in AI](https://mlsanity.com/paper/2505.06464/) (2025)
- [Fully Open Meditron: An Auditable Pipeline for Clinical LLMs](https://mlsanity.com/paper/2605.16215/) (2026)
- [A governance horizon for ethical-use constraints in open-weight AI models](https://mlsanity.com/paper/2605.24383/) (2026)
- [Reproducibility is the New Copyleft: Defining AGI-oriented Reproducible Builds](https://mlsanity.com/paper/2606.03019/) (2026)
- [Position: The Alignment Community is Unintentionally Building a Censor's Toolkit](https://mlsanity.com/paper/2608.12346/) (2026)
- [Traceable Trust for action-ready artificial intelligence in bioscience](https://mlsanity.com/paper/2608.17997/) (2026)
- [A Comprehensive Survey on Linguistic Steganography: Methods, Countermeasures, Evaluation, and Challenges](https://mlsanity.com/paper/2608.29077/) (2026)
- [Language Model Fingerprinting Requires Rethinking Watermark Teachers](https://mlsanity.com/paper/2610.04169/) (2026)

## Abstract (arXiv)

Generative artificial intelligence (AI) offers numerous opportunities for research and innovation, but its commercialization has raised concerns about the transparency and safety of frontier AI models. Most models lack the necessary components for full understanding, auditing, and reproducibility, and some model producers use restrictive licenses whilst claiming that their models are "open source". To address these concerns, we introduce the Model Openness Framework (MOF), a three-tiered ranked classification system that rates machine learning models based on their completeness and openness, following open science principles. For each MOF class, we specify code, data, and documentation components of the model development lifecycle that must be released and under which open licenses. In addition, the Model Openness Tool (MOT) provides a user-friendly reference implementation to evaluate the openness and completeness of models against the MOF classification system. Together, the MOF and MOT provide timely practical guidance for (i) model producers to enhance the openness and completeness of their publicly-released models, and (ii) model consumers to identify open models and their constituent components that can be permissively used, studied, modified, and redistributed. Through the MOF, we seek to establish completeness and openness as core tenets of responsible AI research and development, and to promote best practices in the burgeoning open AI ecosystem.