# Online Learning for Recommendations at Grubhub

arXiv: https://arxiv.org/abs/2107.07106 · HTML: https://mlsanity.com/paper/2107.07106/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 4 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 2 of 2)

- [Model Size Reduction Using Frequency Based Double Hashing for Recommender Systems](https://mlsanity.com/paper/2007.14523/) (2020)
- [Deep Bayesian Bandits: Exploring in Online Personalized Recommendations](https://mlsanity.com/paper/2008.00727/) (2020)

## Built on by (most cited first, 7 of 7)

- [Overcoming the Stability Gap in Continual Learning](https://mlsanity.com/paper/2306.01904/) (2023)
- [Modyn: Data-Centric Machine Learning Pipeline Orchestration](https://mlsanity.com/paper/2312.06254/) (2023)
- [Momentum LMS Theory beyond Stationarity: Stability, Tracking, and Regret](https://mlsanity.com/paper/2602.11995/) (2026)
- [Contextual Bandits in Payment Processing: Non-uniform Exploration and Supervised Learning](https://mlsanity.com/paper/2412.00569/) (2024)
- [Accelerating Recommender Model ETL with a Streaming FPGA-GPU Dataflow](https://mlsanity.com/paper/2501.12032/) (2025)
- [Staying Alive: Online Neural Network Maintenance and Systemic Drift](https://mlsanity.com/paper/2503.17681/) (2025)
- [Balancing Trial and Reorder: A Hybrid Sequential Transformer-GBDT Ranker for On-Demand Delivery](https://mlsanity.com/paper/2609.16407/) (2026)

## Abstract (arXiv)

We propose a method to easily modify existing offline Recommender Systems to run online using Transfer Learning. Online Learning for Recommender Systems has two main advantages: quality and scale. Like many Machine Learning algorithms in production if not regularly retrained will suffer from Concept Drift. A policy that is updated frequently online can adapt to drift faster than a batch system. This is especially true for user-interaction systems like recommenders where the underlying distribution can shift drastically to follow user behaviour. As a platform grows rapidly like Grubhub, the cost of running batch training jobs becomes material. A shift from stateless batch learning offline to stateful incremental learning online can recover, for example, at Grubhub, up to a 45x cost savings and a +20% metrics increase. There are a few challenges to overcome with the transition to online stateful learning, namely convergence, non-stationary embeddings and off-policy evaluation, which we explore from our experiences running this system in production.