# Data and AI governance: Promoting equity, ethics, and fairness in large language models

arXiv: https://arxiv.org/abs/2508.03970 · HTML: https://mlsanity.com/paper/2508.03970/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 15 of its 63 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools](https://mlsanity.com/paper/2405.20362/) (2024)
- [Explaining Hyperparameter Optimization via Partial Dependence Plots](https://mlsanity.com/paper/2111.04820/) (2021)
- [AI Lifecycle Models Need To Be Revised. An Exploratory Study in Fintech](https://mlsanity.com/paper/2010.02716/) (2020)
- [Counterfactual Explanations for Machine Learning: Challenges Revisited](https://mlsanity.com/paper/2106.07756/) (2021)
- [An Artificial Intelligence Life Cycle: From Conception to Production](https://mlsanity.com/paper/2108.13861/) (2021)
- [Fairness Assessment for Artificial Intelligence in Financial Industry](https://mlsanity.com/paper/1912.07211/) (2019)
- [BEATS: Bias Evaluation and Assessment Test Suite for Large Language Models](https://mlsanity.com/paper/2503.24310/) (2025)
- [Navigating Governance Paradigms: A Cross-Regional Comparative Study of Generative AI Governance Processes & Principles](https://mlsanity.com/paper/2408.16771/) (2024)

## Built on by (most cited first, 2 of 2)

- [SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models](https://mlsanity.com/paper/2601.21235/) (2026)
- [FlexMoRE: A Flexible Mixture of Rank-heterogeneous Experts for Efficient Federatedly-trained Large Language Models](https://mlsanity.com/paper/2602.08818/) (2026)

## Abstract (arXiv)

In this paper, we cover approaches to systematically govern, assess and quantify bias across the complete life cycle of machine learning models, from initial development and validation to ongoing production monitoring and guardrail implementation. Building upon our foundational work on the Bias Evaluation and Assessment Test Suite (BEATS) for Large Language Models, the authors share prevalent bias and fairness related gaps in Large Language Models (LLMs) and discuss data and AI governance framework to address Bias, Ethics, Fairness, and Factuality within LLMs. The data and AI governance approach discussed in this paper is suitable for practical, real-world applications, enabling rigorous benchmarking of LLMs prior to production deployment, facilitating continuous real-time evaluation, and proactively governing LLM generated responses. By implementing the data and AI governance across the life cycle of AI development, organizations can significantly enhance the safety and responsibility of their GenAI systems, effectively mitigating risks of discrimination and protecting against potential reputational or brand-related harm. Ultimately, through this article, we aim to contribute to advancement of the creation and deployment of socially responsible and ethically aligned generative artificial intelligence powered applications.