# Online Matching: A Real-time Bandit System for Large-scale Recommendations

arXiv: https://arxiv.org/abs/2307.15893 · HTML: https://mlsanity.com/paper/2307.15893/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 29 Jul 2023.

## Builds on (most cited first, 12 of 12)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [Unbiased Learning-to-Rank with Biased Feedback](https://mlsanity.com/paper/1608.04468/) (2016)
- [Pre-training Tasks for Embedding-based Large-scale Retrieval](https://mlsanity.com/paper/2002.03932/) (2020)
- [Online Learning under Delayed Feedback](https://mlsanity.com/paper/1306.0686/) (2013)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Offline A/B testing for Recommender Systems](https://mlsanity.com/paper/1801.07030/) (2018)
- [Monolith: Real Time Recommendation System With Collisionless Embedding Table](https://mlsanity.com/paper/2209.07663/) (2022)
- [Related Pins at Pinterest: The Evolution of a Real-World Recommender System](https://mlsanity.com/paper/1702.07969/) (2017)
- [Linear Bandits in High Dimension and Recommendation Systems](https://mlsanity.com/paper/1301.1722/) (2013)
- [Visual Discovery at Pinterest](https://mlsanity.com/paper/1702.04680/) (2017)
- [Show Me the Whole World: Towards Entire Item Space Exploration for Interactive Personalized Recommendations](https://mlsanity.com/paper/2110.09905/) (2021)

## Built on by (most cited first, 8 of 8)

- [Optimal Baseline Corrections for Off-Policy Contextual Bandits](https://mlsanity.com/paper/2405.05736/) (2024)
- [Beyond Item Dissimilarities: Diversifying by Intent in Recommender Systems](https://mlsanity.com/paper/2405.12327/) (2024)
- [Scalable LinUCB: Low-Rank Design Matrix Updates for Recommenders with Large Action Spaces](https://mlsanity.com/paper/2510.19349/) (2025)
- [Causal Feature Selection Method for Contextual Multi-Armed Bandits in Recommender System](https://mlsanity.com/paper/2409.13888/) (2024)
- [RADAR: Recall Augmentation through Deferred Asynchronous Retrieval](https://mlsanity.com/paper/2506.07261/) (2025)
- [Dynamic Prior Thompson Sampling for Cold-Start Exploration in Recommender Systems](https://mlsanity.com/paper/2602.00943/) (2026)
- [Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest](https://mlsanity.com/paper/2607.14161/) (2026)
- [Designing for the Next Click: Bandits for Real-Time Page Layout](https://mlsanity.com/paper/2608.29850/) (2026)

## Abstract (arXiv)

The last decade has witnessed many successes of deep learning-based models for industry-scale recommender systems. These models are typically trained offline in a batch manner. While being effective in capturing users' past interactions with recommendation platforms, batch learning suffers from long model-update latency and is vulnerable to system biases, making it hard to adapt to distribution shift and explore new items or user interests. Although online learning-based approaches (e.g., multi-armed bandits) have demonstrated promising theoretical results in tackling these challenges, their practical real-time implementation in large-scale recommender systems remains limited. First, the scalability of online approaches in servicing a massive online traffic while ensuring timely updates of bandit parameters poses a significant challenge. Additionally, exploring uncertainty in recommender systems can easily result in unfavorable user experience, highlighting the need for devising intricate strategies that effectively balance the trade-off between exploitation and exploration. In this paper, we introduce Online Matching: a scalable closed-loop bandit system learning from users' direct feedback on items in real time. We present a hybrid "offline + online" approach for constructing this system, accompanied by a comprehensive exposition of the end-to-end system architecture. We propose Diag-LinUCB -- a novel extension of the LinUCB algorithm -- to enable distributed updates of bandits parameter in a scalable and timely manner. We conduct live experiments in YouTube and show that Online Matching is able to enhance the capabilities of fresh content discovery and item exploration in the present platform.