# SmartChoices: Augmenting Software with Learned Implementations

arXiv: https://arxiv.org/abs/2304.13033 · HTML: https://mlsanity.com/paper/2304.13033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability](https://mlsanity.com/paper/2003.12699/) (2020)
- [Stochastic Optimization of Sorting Networks via Continuous Relaxations](https://mlsanity.com/paper/1903.08850/) (2019)
- [Bayesian Optimization with Unknown Constraints](https://mlsanity.com/paper/1403.5607/) (2014)
- [Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimization](https://mlsanity.com/paper/2006.04655/) (2020)
- [Horizon: Facebook's Open Source Applied Reinforcement Learning Platform](https://mlsanity.com/paper/1811.00260/) (2018)
- [An efficient algorithm for contextual bandits with knapsacks, and an extension to concave objectives](https://mlsanity.com/paper/1506.03374/) (2015)
- [Contextual Multi-Armed Bandits for Causal Marketing](https://mlsanity.com/paper/1810.01859/) (2018)
- [Learning to Rank in the Position Based Model with Bandit Feedback](https://mlsanity.com/paper/2004.13106/) (2020)
- [Looper: An end-to-end ML platform for product decisions](https://mlsanity.com/paper/2110.07554/) (2021)
- [Contextual Bandit Applications in Customer Support Bot](https://mlsanity.com/paper/2112.03210/) (2021)
- [Smart Choices and the Selection Monad](https://mlsanity.com/paper/2007.08926/) (2020)
- [Programming by Rewards](https://mlsanity.com/paper/2007.06835/) (2020)

## Abstract (arXiv)

In many software systems, heuristics are used to make decisions - such as cache eviction, task scheduling, and information presentation - that have a significant impact on overall system behavior. While machine learning may outperform these heuristics, replacing existing heuristics in a production system safely and reliably can be prohibitively costly. We present SmartChoices, a novel approach that reduces the cost to deploy production-ready ML solutions for contextual bandits problems. SmartChoices' interface cleanly separates problem formulation from implementation details: engineers describe their use case by defining datatypes for the context, arms, and feedback that are passed to SmartChoices APIs, while SmartChoices manages encoding & logging data and training, evaluating & deploying policies. Our implementation codifies best practices, is efficient enough for use in low-level applications, and provides valuable production features off the shelf via a shared library. Overall, SmartChoices enables non-experts to rapidly deploy production-ready ML solutions by eliminating many sources of technical debt common to ML systems. Engineers have independently used SmartChoices to improve a wide range of software including caches, batch processing workloads, and UI layouts, resulting in better latency, throughput, and click-through rates.