# Specification-delta-driven data governance: an empirical study of the guillemotleftspec-deltaguillemotright as the unit of change in lakehouse data platforms

arXiv: https://arxiv.org/abs/2608.19838 · HTML: https://mlsanity.com/paper/2608.19838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 5 of 5)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems](https://mlsanity.com/paper/2203.10384/) (2022)
- [AI-Driven Generation of Data Contracts in Modern Data Engineering Systems](https://mlsanity.com/paper/2507.21056/) (2025)
- [Constitutional Spec-Driven Development: Enforcing Security by Construction in AI-Assisted Code Generation](https://mlsanity.com/paper/2602.02584/) (2026)

## Abstract (arXiv)

Spec Driven Development SDD has consolidated the idea that the specification rather than the code should be the primary artefact governing AI assisted work. Tools such as GitHub Spec Kit, and proposals such as Constitutional SDD, have formalised this principle in the software domain, while the executable data-contracts literature has extended it to schema and quality enforcement at run time. Nevertheless, the treatment of the specification delta OpenSpec's core idea that every change should produce a reviewable increment of requirements as the unit of change in data platforms remains empirically unexplored, even though many data-platform changes are contractual (new datasets, service-level agreements, metric semantics, access policies) rather than purely code changes. This work formalises the spec-delta concept, proposes a taxonomy of data platform changes according to their suitability for incremental specification, and defines a controlled experiment comparing a spec-delta-driven workflow against a conventional code pull-request workflow without a delta. The response variables are discovery to deployment time, the density of defects reaching the Silver and Gold lakehouse layers, cross-tool metric divergence, and reviewer cognitive load measured with NASA TLX. The paper explicitly reserves a demonstration-and-laboratory section for instantiation on a real lakehouse environment. The contribution is not a tool but reproducible evidence and an applicability guide that helps to avoid the up front over specification antipattern.