mlsanityData to 7 Oct 2026

Reducing Dueling Bandits to Cardinal Bandits

arXiv 1405.3396 ↗First posted 14 May 2014cs.LGcited by 62

Not on the map. Why?

Abstract (arXiv)

We present algorithms for reducing the Dueling Bandits problem to the conventional (stochastic) Multi-Armed Bandits problem. The Dueling Bandits problem is an online model of learning with ordinal feedback of the form "A is preferred to B" (as opposed to cardinal feedback like "A has value 2.5"), giving it wide applicability in learning from implicit user feedback and revealed and stated preferences. In contrast to existing algorithms for the Dueling Bandits problem, our reductions -- named Doubler, MultiSbm and DoubleSbm -- provide a generic schema for translating the extensive body of known results about conventional Multi-Armed Bandit algorithms to the Dueling Bandits setting. For Doubler and MultiSbm we prove regret upper bounds in both finite and infinite settings, and conjecture about the performance of DoubleSbm which empirically outperforms the other two as well as previous algorithms in our experiments. In addition, we provide the first almost optimal regret bound in terms of second order terms, such as the differences between the values of the arms.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 10; 2023: 10; 2024: 14; 2025: 13; 2026: 8.

62 papers.

Sort:

By area:

  1. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 244
  2. 2021Dueling RL: Reinforcement Learning with Trajectory Preferencescited by 105
  3. 2023Is RLHF More Difficult than Standard RL?cited by 96
  4. 2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
  5. 2021Efficient and Optimal Algorithms for Contextual Dueling Bandits under Realizabilitycited by 44
  6. 2022Stochastic Contextual Dueling Bandits under Linear Stochastic Transitivity Modelscited by 35
  7. 2024Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferencescited by 27
  8. 2023Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problemscited by 12
  9. 2024Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inferencecited by 12
  10. 2022Exploiting Correlation to Achieve Faster Learning Rates in Low-Rank Preference Banditscited by 11
  11. 2022Batched Dueling Banditscited by 10
  12. 2021Preference learning along multiple criteria: A game-theoretic perspectivecited by 9
38 more
  1. 2021Optimal and Efficient Dynamic Regret Algorithms for Non-Stationary Dueling Banditscited by 9
  2. 2024Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedbackcited by 9
  3. 2025Instance-Dependent Regret Bounds for Learning Two-Player Zero-Sum Games with Bandit Feedbackcited by 9
  4. 2025ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignmentcited by 9
  5. 2022ANACONDA: An Improved Dynamic Regret Algorithm for Adaptive Non-Stationary Dueling Banditscited by 8
  6. 2023Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedbackcited by 7
  7. 2024Bandits with Preference Feedback: A Stackelberg Game Perspectivecited by 7
  8. 2024Neural Dueling Bandits: Preference-Based Optimization with Human Feedbackcited by 7
  9. 2023Active Ranking of Experts Based on their Performances in Many Taskscited by 6
  10. 2021Dueling Bandits with Adversarial Sleepingcited by 5
  11. 2022Versatile Dueling Bandits: Best-of-both-World Analyses for Online Learning from Preferencescited by 5
  12. 2022Dueling Convex Optimization with General Preferencescited by 5
  13. 2023When Can We Track Significant Preference Shifts in Dueling Bandits?cited by 5
  14. 2022Dueling Bandits: From Two-dueling to Multi-duelingcited by 4
  15. 2024Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysiscited by 4
  16. 2025Active Human Feedback Collection via Neural Contextual Dueling Banditscited by 4
  17. 2022Learning from a Learning User for Optimal Recommendationscited by 3
  18. 2022One Arrow, Two Kills: An Unified Framework for Achieving Optimal Regret Guarantees in Sleeping Banditscited by 3
  19. 2023Combinatorial Bandits for Maximum Value Reward Function under Max Value-Index Feedbackcited by 3
  20. 2023Think Before You Duel: Understanding Complexities of Preference Learning under Constrained Resourcescited by 3
  21. 2024Stop Relying on No-Choice and Do not Repeat the Moves: Optimal, Efficient and Practical Algorithms for Assortment Optimizationcited by 3
  22. 2024Biased Dueling Bandits with Stochastic Delayed Feedbackcited by 3
  23. 2025Bayesian Optimization from Human Feedback: Near-Optimal Regret Boundscited by 3
  24. 2025Efficient Preference-Based Reinforcement Learning: Randomized Exploration Meets Experimental Designcited by 3
  25. 2022An Asymptotically Optimal Batched Algorithm for the Dueling Bandit Problemcited by 2
  26. 2023Faster Convergence with Multiway Preferencescited by 2
  27. 2024Non-Stationary Dueling Bandits Under a Weighted Borda Criterioncited by 2
  28. 2024Learning from Imperfect Human Feedback: a Tale from Corruption-Robust Duelingcited by 2
  29. 2024Beyond Numeric Rewards: In-Context Dueling Bandits with LLM Agentscited by 2
  30. 2025Federated Linear Dueling Banditscited by 2
  31. 2025Online Clustering of Dueling Banditscited by 2
  32. 2025Clustering Items through Bandit Feedback: Finding the Right Feature out of Manycited by 2
  33. 2025Reinforcement Learning from Adversarial Preferences in Tabular MDPscited by 2
  34. 2026Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferencescited by 2
  35. 2021Combinatorial Bandits without Total Order for Armscited by 1
  36. 2024DP-Dueling: Learning from Preference Feedback without Compromising User Privacycited by 1
  37. 2024The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedbackcited by 1
  38. 2025Fusing Reward and Dueling Feedback in Stochastic Banditscited by 1

50 most cited of 62 All 62 (JSON)