# Toward Trustworthy Evaluation of Sustainability Rating Methodologies: A Human-AI Collaborative Framework for Benchmark Dataset Construction

arXiv: https://arxiv.org/abs/2602.17106 · HTML: https://mlsanity.com/paper/2602.17106/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 90 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 21 of 21)

- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle](https://mlsanity.com/paper/1901.10002/) (2019)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [Mapping global dynamics of benchmark creation and saturation in artificial intelligence](https://mlsanity.com/paper/2203.04592/) (2022)
- [Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems](https://mlsanity.com/paper/2102.04201/) (2021)
- [The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems](https://mlsanity.com/paper/2509.08713/) (2025)
- [ESGReveal: An LLM-based approach for extracting structured data from ESG reports](https://mlsanity.com/paper/2312.17264/) (2023)
- [Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI](https://mlsanity.com/paper/2309.12327/) (2023)
- [ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge](https://mlsanity.com/paper/2506.01646/) (2025)
- [Glitter or Gold? Deriving Structured Insights from Sustainability Reports via Large Language Models](https://mlsanity.com/paper/2310.05628/) (2023)
- [Collect, Measure, Repeat: Reliability Factors for Responsible AI Data Collection](https://mlsanity.com/paper/2308.12885/) (2023)
- [CleanGraph: Human-in-the-loop Knowledge Graph Refinement and Completion](https://mlsanity.com/paper/2405.03932/) (2024)
- [Structuring the Unstructured: A Multi-Agent System for Extracting and Querying Financial KPIs and Guidance](https://mlsanity.com/paper/2505.19197/) (2025)
- [Evaluating machine learning models in non-standard settings: An overview and new findings](https://mlsanity.com/paper/2310.15108/) (2023)
- [Human and AI Trust: Trust Attitude Measurement Instrument](https://mlsanity.com/paper/2510.21535/) (2025)

## Built on by (most cited first, 1 of 1)

- [Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle](https://mlsanity.com/paper/2607.22877/) (2026)

## Abstract (arXiv)

Sustainability or ESG rating agencies use company disclosures and external data to produce scores or ratings that assess the environmental, social, and governance performance of a company. However, sustainability ratings across agencies for a single company vary widely, limiting their comparability, credibility, and relevance to decision-making. To harmonize the rating results, we propose adopting a universal human-AI collaboration framework to generate trustworthy benchmark datasets for evaluating sustainability rating methodologies. The framework comprises two complementary parts: STRIDE (Sustainability Trust Rating & Integrity Data Equation) provides principled criteria and a scoring system that guide the construction of firm-level benchmark datasets using large language models (LLMs), and SR-Delta, a discrepancy-analysis procedural framework that surfaces insights for potential adjustments. The framework enables scalable and comparable assessment of sustainability rating methodologies. We call on the broader AI community to adopt AI-powered approaches to strengthen and advance sustainability rating methodologies that support and enforce urgent sustainability agendas.