mlsanityData to 14 Oct 2026

Generative Responsible AI Data Evaluation Schema (GRAIDES) for AI Assurance in Local Government

arXiv 2606.20963 ↗First posted 18 Jun 2026cs.AI cs.CY cs.DBcited by 0

Not on the map. Why?

Abstract (arXiv)

Trust in the application of generative Artificial Intelligence (AI) relies on well-governed measurable evidence of performance and safety. In practice, however, evaluation data is often fragmented across systems, inconsistently structured and difficult to compare. We introduce the Generative Responsible AI Data Evaluation Schema (GRAIDES) as a lightweight open-source data model for centralising AI observability across popular vendors. Practical blueprints for code, architecture and statistical evaluation are shared as guidance about how to approach generative system assurance at the organisational level. Illustrative case study results are reported from Westminster City Council's AI catalogue with a focus on measuring human-model alignment including detecting systematic disagreement between evaluators. By framing evaluations as a data modelling problem, GRAIDES provides a practical pathway toward more consistent and reproducible benchmarking, tuning and assurance activities for generative AI systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 25 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

10 of 25 references.

References · 10

  1. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  2. 2024A Survey on LLM-as-a-Judgecited by 1,423
  3. 2021Ethical and social risks of harm from Language Modelscited by 966
  4. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 205
  5. 2022Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineeringcited by 20
  6. 2026Learning to Judge: LLMs Designing and Applying Evaluation Rubricscited by 19
  7. 2025HiBayES: A Hierarchical Bayesian Modeling Framework for AI Evaluation Statisticscited by 18
  8. 2025Don't Pass@k: A Bayesian Framework for Large Language Model Evaluationcited by 15
  9. 2025Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systemscited by 14
  10. 2024Ethical and Scalable Automation: A Governance and Compliance Framework for Business Applicationscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Who is Responsible? The Data, Models, Users or Regulations? A Comprehensive Survey on Responsible Generative AI for a Sustainable Future
  2. 2025SAIF: A Comprehensive Framework for Evaluating the Risks of Generative AI in the Public Sector
  3. 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Era
  4. 2026EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
  5. 2024A Safe Harbor for AI Evaluation and Red Teaming
  6. 2026NodeSynth: Socially Aligned Synthetic Data for AI Evaluation
  7. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Review
  8. 2026A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems