Data to 14 Oct 2026

Improved Optimistic Algorithms for Logistic Bandits

arXiv 2002.07530 ↗First posted 18 Feb 2020cs.LG stat.MLcited by 118

Not on the map. Why?

Cited by 22 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The generalized linear bandit framework has attracted a lot of attention in recent years by extending the well-understood linear setting and allowing to model richer reward structures. It notably covers the logistic model, widely used when rewards are binary. For logistic bandits, the frequentist regret guarantees of existing algorithms are tildeO(kappa √T), where kappa is a problem-dependent constant. Unfortunately, kappa can be arbitrarily large as it scales exponentially with the size of the decision set. This may lead to significantly loose regret bounds and poor empirical performance. In this work, we study the logistic bandit with a focus on the prohibitive dependencies introduced by kappa. We propose a new optimistic algorithm based on a finer examination of the non-linearities of the reward function. We show that it enjoys a tildeO(√T) regret with no dependency in kappa, but for a second order term. Our analysis is based on a new tail-inequality for self-normalized martingales, of independent interest.

Key facts. Cited by 118 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 11; 2024: 25; 2025: 32; 2026: 25.

118 papers.

Sort:

By area:

2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 350
2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
2024DPO Meets PPO: Reinforced Token Optimization for RLHFcited by 127
2021Dueling RL: Reinforcement Learning with Trajectory Preferencescited by 106
2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
2021Improved Variance-Aware Confidence Sets for Linear Bandits and Linear Mixture MDPcited by 45
2022Jointly Efficient and Optimal Algorithms for Logistic Banditscited by 41
2021Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvaturecited by 32
2021On the Theory of Reinforcement Learning with Once-per-Episode Feedbackcited by 32
2025Language Model Personalization via Reward Factorizationcited by 31
2022Nearly Minimax Optimal Reinforcement Learning with Linear Function Approximationcited by 30
2024A Unified Confidence Sequence for Generalized Linear Models, with Applications to Banditscited by 29

38 more

2024Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferencescited by 27
2024Optimal Design for Reward Modeling in RLHFcited by 25
2023Improved Regret Bounds of (Multinomial) Logistic Bandits via Regret-to-Confidence-Set Conversioncited by 24
2022Delayed Feedback in Generalised Linear Bandits Revisitedcited by 23
2024Generalized Linear Bandits with Limited Adaptivitycited by 22
2021Dynamic pricing and assortment under a contextual MNL demandcited by 20
2024Nearly Minimax Optimal Regret for Multinomial Logistic Banditcited by 20
2022Lifting the Information Ratio: An Information-Theoretic Analysis of Thompson Sampling for Contextual Banditscited by 19
2022Learning Neural Contextual Bandits Through Perturbed Rewardscited by 17
2023Borda Regret Minimization for Generalized Linear Dueling Banditscited by 17
2025Generalized Linear Bandits: Almost Optimal Regret with One-Pass Updatecited by 16
2022Fast Rates in Pool-Based Batch Active Learningcited by 15
2025Improved Online Confidence Bounds for Multinomial Logistic Banditscited by 15
2022Double Doubly Robust Thompson Sampling for Generalized Linear Contextual Banditscited by 14
2021Regret Bounds for Generalized Linear Bandits under Parameter Driftcited by 13
2022Bregman Deviations of Generic Exponential Familiescited by 13
2023Exploration via linearly perturbed loss minimisationcited by 13
2024Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Driftcited by 13
2021UCB-based Algorithms for Multinomial Logistic Regression Banditscited by 12
2022Variance-Aware Sparse Linear Banditscited by 11
2023Likelihood Ratio Confidence Sets for Sequential Decision Makingcited by 10
2024Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximationcited by 10
2024Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedbackcited by 9
2024Advances in Preference-based Reinforcement Learning: A Reviewcited by 8
2024Almost Free: Self-concordance in Natural Exponential Families and an Application to Banditscited by 8
2025Avoiding mathbfexp(Rmax) scaling in RLHF through Preference-based Explorationcited by 8
2022An Experimental Design Approach for Regret Minimization in Logistic Banditscited by 7
2024Randomized Exploration for Reinforcement Learning with Multinomial Logistic Function Approximationcited by 7
2024Bandits with Preference Feedback: A Stackelberg Game Perspectivecited by 7
2024Neural Dueling Bandits: Preference-Based Optimization with Human Feedbackcited by 7
2023Time-Uniform Confidence Spheres for Means of Random Vectorscited by 6
2024Improved Algorithm for Adversarial Linear Mixture MDPs with Bandit Feedback and Unknown Transitioncited by 6
2024Horizon-Free Regret for Linear Markov Decision Processescited by 6
2024Active Preference Learning for Ordering Items In- and Out-of-samplecited by 6
2025Neural Logistic Banditscited by 6
2025Inference-Time Personalized Alignment with a Few User Preference Queriescited by 6
2026Coverage Improvement and Fast Convergence of On-policy Preference Learningcited by 6
2026Revisiting Weighted Strategy for Non-stationary Parametric Bandits and MDPscited by 5

50 most cited of 118 All 118 (JSON)