# Overton: A Data System for Monitoring and Improving Machine-Learned Products

arXiv: https://arxiv.org/abs/1909.05372 · HTML: https://mlsanity.com/paper/1909.05372/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2019.

## Built on by (most cited first, 21 of 21)

- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [A Data Quality-Driven View of MLOps](https://mlsanity.com/paper/2102.07750/) (2021)
- [Universalizing Weak Supervision](https://mlsanity.com/paper/2112.03865/) (2021)
- [Management of Machine Learning Lifecycle Artifacts: A Survey](https://mlsanity.com/paper/2210.11831/) (2022)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [Lifting Weak Supervision To Structured Prediction](https://mlsanity.com/paper/2211.13375/) (2022)
- [Looper: An end-to-end ML platform for product decisions](https://mlsanity.com/paper/2110.07554/) (2021)
- [Predictive Query Language: A Domain-Specific Language for Predictive Modeling on Relational Databases](https://mlsanity.com/paper/2602.09572/) (2026)
- [Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins](https://mlsanity.com/paper/2106.01501/) (2021)
- [AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels](https://mlsanity.com/paper/2208.14362/) (2022)
- [Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems](https://mlsanity.com/paper/2108.05053/) (2021)
- [Maintainability Challenges in ML: A Systematic Literature Review](https://mlsanity.com/paper/2408.09196/) (2024)
- [Declarative Machine Learning Systems](https://mlsanity.com/paper/2107.08148/) (2021)
- [Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach](https://mlsanity.com/paper/2410.21346/) (2024)
- [AutoML in Heavily Constrained Applications](https://mlsanity.com/paper/2306.16913/) (2023)
- [OpenJarvis: Personal AI, On Personal Devices](https://mlsanity.com/paper/2605.17172/) (2026)
- [Fantastic Data and How to Query Them](https://mlsanity.com/paper/2201.05026/) (2022)
- [Leveraging Large Language Models for Structure Learning in Prompted Weak Supervision](https://mlsanity.com/paper/2402.01867/) (2024)
- [Generative Modeling Helps Weak Supervision (and Vice Versa)](https://mlsanity.com/paper/2203.12023/) (2022)
- [Scalability and Maintainability Challenges and Solutions in Machine Learning: Systematic Literature Review](https://mlsanity.com/paper/2504.11079/) (2025)

## Abstract (arXiv)

We describe a system called Overton, whose main design goal is to support engineers in building, monitoring, and improving production machine learning systems. Key challenges engineers face are monitoring fine-grained quality, diagnosing errors in sophisticated applications, and handling contradictory or incomplete supervision data. Overton automates the life cycle of model construction, deployment, and monitoring by providing a set of novel high-level, declarative abstractions. Overton's vision is to shift developers to these higher-level tasks instead of lower-level machine learning tasks. In fact, using Overton, engineers can build deep-learning-based applications without writing any code in frameworks like TensorFlow. For over a year, Overton has been used in production to support multiple applications in both near-real-time applications and back-of-house processing. In that time, Overton-based applications have answered billions of queries in multiple languages and processed trillions of records reducing errors 1.7-2.9 times versus production systems.