# Linear Contextual Bandits with Interference

arXiv: https://arxiv.org/abs/2409.15682 · HTML: https://mlsanity.com/paper/2409.15682/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 14 of 14)

- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [Confidence Intervals for Policy Evaluation in Adaptive Experiments](https://mlsanity.com/paper/1911.02768/) (2019)
- [Inference for Batched Bandits](https://mlsanity.com/paper/2002.03217/) (2020)
- [Identification and estimation of treatment and interference effects in observational studies on networks](https://mlsanity.com/paper/1609.06245/) (2016)
- [Accurate Inference for Adaptive Linear Models](https://mlsanity.com/paper/1712.06695/) (2017)
- [On Distributed Cooperative Decision-Making in Multiarmed Bandits](https://mlsanity.com/paper/1512.06888/) (2015)
- [Kernel Methods for Cooperative Multi-Agent Contextual Bandits](https://mlsanity.com/paper/2008.06220/) (2020)
- [Statistical Inference for Online Decision-Making: In a Contextual Bandit Setting](https://mlsanity.com/paper/2010.07283/) (2020)
- [Rate-Optimal Cluster-Randomized Designs for Spatial Interference](https://mlsanity.com/paper/2111.04219/) (2021)
- [Heterogeneous Treatment and Spillover Effects under Clustered Network Interference](https://mlsanity.com/paper/2008.00707/) (2020)
- [Multi-Armed Bandits with Interference](https://mlsanity.com/paper/2402.01845/) (2024)
- [Experimental Design under Network Interference](https://mlsanity.com/paper/2003.08421/) (2020)
- [Multi-Armed Bandits with Network Interference](https://mlsanity.com/paper/2405.18621/) (2024)
- [Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learning](https://mlsanity.com/paper/2110.15501/) (2021)

## Built on by (most cited first, 8 of 8)

- [Online Experimental Design With Estimation-Regret Trade-off Under Network Interference](https://mlsanity.com/paper/2412.03727/) (2024)
- [Graph-Dependent Regret Bounds in Multi-Armed Bandits with Interference](https://mlsanity.com/paper/2503.07555/) (2025)
- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [Scalable Policy Maximization Under Network Interference](https://mlsanity.com/paper/2505.18118/) (2025)
- [Design-Based Bandits Under Network Interference: Trade-Off Between Regret and Statistical Inference](https://mlsanity.com/paper/2510.07646/) (2025)
- [PARWiS: Winner determination under shoestring budgets using active pairwise comparisons](https://mlsanity.com/paper/2603.01171/) (2026)
- [Tighter Regret Bounds for Contextual Action-Set Reinforcement Learning](https://mlsanity.com/paper/2605.15692/) (2026)
- [Learning to target with network interference](https://mlsanity.com/paper/2605.27794/) (2026)

## Abstract (arXiv)

Interference, a key concept in causal inference, extends the reward modeling process by accounting for the impact of one unit's actions on the rewards of others. In contextual bandit (CB) settings, where multiple units are present in the same round, potential interference can significantly affect the estimation of expected rewards for different arms, thereby influencing the decision-making process. Although some prior work has explored multi-agent and adversarial bandits in interference-aware settings, the effect of interference in CB, as well as the underlying theory, remains significantly underexplored. In this paper, we introduce a systematic framework to address interference in Linear CB (LinCB), bridging the gap between causal inference and online decision-making. We propose a series of algorithms that explicitly quantify the interference effect in the reward modeling process and provide comprehensive theoretical guarantees, including sublinear regret bounds, finite sample upper bounds, and asymptotic properties. The effectiveness of our approach is demonstrated through simulations and a synthetic data generated based on MovieLens data.