# What Kinds of Contracts Do ML APIs Need?

arXiv: https://arxiv.org/abs/2307.14465 · HTML: https://mlsanity.com/paper/2307.14465/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 79 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 8 of 8)

- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [A Composable Specification Language for Reinforcement Learning Tasks](https://mlsanity.com/paper/2008.09293/) (2020)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020)
- [DocTer: Documentation Guided Fuzzing for Testing Deep Learning API Functions](https://mlsanity.com/paper/2109.01002/) (2021)
- [Interpreting Cloud Computer Vision Pain-Points: A Mining Study of Stack Overflow](https://mlsanity.com/paper/2001.10130/) (2020)
- [Automated Fixing of Programs with Contracts](https://mlsanity.com/paper/1403.1117/) (2014)
- [DARVIZ: Deep Abstract Representation, Visualization, and Verification of Deep Learning Models](https://mlsanity.com/paper/1708.04915/) (2017)

## Built on by (most cited first, 2 of 2)

- [Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment](https://mlsanity.com/paper/2401.14628/) (2024)
- [Data-aware Static Analysis: Improving Detection of Semantic Faults in Machine Learning Code Using Data Characteristics](https://mlsanity.com/paper/2606.09957/) (2026)

## Abstract (arXiv)

Recent work has shown that Machine Learning (ML) programs are error-prone and called for contracts for ML code. Contracts, as in the design by contract methodology, help document APIs and aid API users in writing correct code. The question is: what kinds of contracts would provide the most help to API users? We are especially interested in what kinds of contracts help API users catch errors at earlier stages in the ML pipeline. We describe an empirical study of posts on Stack Overflow of the four most often-discussed ML libraries: TensorFlow, Scikit-learn, Keras, and PyTorch. For these libraries, our study extracted 413 informal (English) API specifications. We used these specifications to understand the following questions. What are the root causes and effects behind ML contract violations? Are there common patterns of ML contract violations? When does understanding ML contracts require an advanced level of ML software expertise? Could checking contracts at the API level help detect the violations in early ML pipeline stages? Our key findings are that the most commonly needed contracts for ML APIs are either checking constraints on single arguments of an API or on the order of API calls. The software engineering community could employ existing contract mining approaches to mine these contracts to promote an increased understanding of ML APIs. We also noted a need to combine behavioral and temporal contract mining approaches. We report on categories of required ML contracts, which may help designers of contract languages.