# Long-Term Value of Exploration: Measurements, Findings and Algorithms

arXiv: https://arxiv.org/abs/2305.07764 · HTML: https://mlsanity.com/paper/2305.07764/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2023.

## Built on by (most cited first, 13 of 13)

- [LLMs for User Interest Exploration in Large-scale Recommendation Systems](https://mlsanity.com/paper/2405.16363/) (2024)
- [Multi-Armed Bandits With Machine Learning-Generated Surrogate Rewards](https://mlsanity.com/paper/2506.16658/) (2025)
- [Optimal Baseline Corrections for Off-Policy Contextual Bandits](https://mlsanity.com/paper/2405.05736/) (2024)
- [Multi-Objective Recommendation via Multivariate Policy Learning](https://mlsanity.com/paper/2405.02141/) (2024)
- [Non-Stationary Contextual Bandit Learning via Neural Predictive Ensemble Sampling](https://mlsanity.com/paper/2310.07786/) (2023)
- [BanditLP: Large-Scale Stochastic Optimization for Personalized Recommendations](https://mlsanity.com/paper/2601.15552/) (2026)
- [PinEqualizer: Full Funnel Content Exploration and Debiasing System at Pinterest](https://mlsanity.com/paper/2607.22518/) (2026)
- [The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing](https://mlsanity.com/paper/2608.04432/) (2026)
- [When Relevance Meets Novelty: Dual-Stable Periodic Optimization for Serendipitous Recommendation](https://mlsanity.com/paper/2508.00450/) (2025)
- [Optimality of Sub-network Laplace Approximations: New Results and Methods](https://mlsanity.com/paper/2605.09075/) (2026)
- [FunnelAL: Retrieve-then-Rank Active Learning for Single-Class Discovery](https://mlsanity.com/paper/2607.25276/) (2026)
- [From Prediction to Incrementality: Causal Optimization for Large-Scale Targeting and Recommendation](https://mlsanity.com/paper/2608.10182/) (2026)
- [Balancing Trial and Reorder: A Hybrid Sequential Transformer-GBDT Ranker for On-Demand Delivery](https://mlsanity.com/paper/2609.16407/) (2026)

## Abstract (arXiv)

Effective exploration is believed to positively influence the long-term user experience on recommendation platforms. Determining its exact benefits, however, has been challenging. Regular A/B tests on exploration often measure neutral or even negative engagement metrics while failing to capture its long-term benefits. We here introduce new experiment designs to formally quantify the long-term value of exploration by examining its effects on content corpus, and connecting content corpus growth to the long-term user experience from real-world experiments. Once established the values of exploration, we investigate the Neural Linear Bandit algorithm as a general framework to introduce exploration into any deep learning based ranking systems. We conduct live experiments on one of the largest short-form video recommendation platforms that serves billions of users to validate the new experiment designs, quantify the long-term values of exploration, and to verify the effectiveness of the adopted neural linear bandit algorithm for exploration.