mlsanityData to 14 Oct 2026

Survey: Multi-Armed Bandits Meet Large Language Models

arXiv 2505.13355 ↗First posted 19 May 2025cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Bandit algorithms and Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, each addressing distinct yet complementary challenges in decision-making and natural language processing. This survey explores the synergistic potential between these two fields, highlighting how bandit algorithms can enhance the performance of LLMs and how LLMs, in turn, can provide novel insights for improving bandit-based decision-making. We first examine the role of bandit algorithms in optimizing LLM fine-tuning, prompt engineering, and adaptive response generation, focusing on their ability to balance exploration and exploitation in large-scale learning tasks. Subsequently, we explore how LLMs can augment bandit algorithms through advanced contextual understanding, dynamic adaptation, and improved policy selection using natural language reasoning. By providing a comprehensive review of existing research and identifying key challenges and opportunities, this survey aims to bridge the gap between bandit algorithms and LLMs, paving the way for innovative applications and interdisciplinary research in AI.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 45 of its 71 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

45 of 71 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  6. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 264
  7. 2017Automated Curriculum Learning for Neural Networkscited by 257
  8. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  9. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 175
  10. 2023Matching Patients to Clinical Trials with Large Language Modelscited by 69
  11. 2025Reinforcement Learning from Human Feedbackcited by 58
  12. 2023Learning to Generate Better Than Your LLMcited by 49
33 more
  1. 2019An ADMM Based Framework for AutoML Pipeline Configurationcited by 47
  2. 2024StreamBench: Towards Benchmarking Continuous Improvement of Language Agentscited by 47
  3. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 37
  4. 2021Hierarchical Bayesian Banditscited by 35
  5. 2025LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routingcited by 34
  6. 2017Scalable Recollections for Continual Lifelong Learningcited by 33
  7. 2024Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Banditscited by 27
  8. 2024Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysiscited by 21
  9. 2018Incorporating Behavioral Constraints in Online AI Systemscited by 16
  10. 2025The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generationcited by 16
  11. 2017Context Attentive Bandits: Contextual Bandit with Restricted Contextcited by 14
  12. 2024Generative AI for the Optimization of Next-Generation Wireless Networks: Basics, State-of-the-Art, and Open Challengescited by 14
  13. 2019A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatrycited by 13
  14. 2020Adam with Bandit Sampling for Deep Learningcited by 11
  15. 2022Non-Stationary Bandits with Auto-Regressive Temporal Dependencycited by 10
  16. 2023LLMs-augmented Contextual Banditcited by 8
  17. 2024Personalized Adaptation via In-Context Preference Learningcited by 8
  18. 2025A Multi-Agent Conversational Bandit Approach to Online Evaluation and Selection of User-Aligned LLM Responsescited by 8
  19. 2020Online Learning in Iterated Prisoner's Dilemma to Mimic Human Behaviorcited by 6
  20. 2023Psychotherapy AI Companion with Reinforcement Learning Recommendations and Interpretable Policy Dynamicscited by 6
  21. 2021Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budgetcited by 5
  22. 2023Ensemble Active Learning by Contextual Bandits for AI Incubation in Manufacturingcited by 5
  23. 2024Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thoughtcited by 5
  24. 2024Online Personalizing White-box LLMs Generation with Neural Banditscited by 5
  25. 2023Interpolating Item and User Fairness in Multi-Sided Recommendationscited by 3
  26. 2024AQA: Adaptive Question Answering in a Society of LLMs via Contextual Multi-Armed Banditcited by 3
  27. 2021Reinforcement Learning with Algorithms from Probabilistic Structure Estimationcited by 2
  28. 2020Adaptive strategy in differential evolution via explicit exploitation and exploration controlscited by 1
  29. 2020Double-Linear Thompson Sampling for Context-Attentive Banditscited by 1
  30. 2022Targeted Advertising on Social Networks Using Online Variational Tensor Regressioncited by 1
  31. 2023Bandit-Driven Batch Selection for Robust Learning under Label Noisecited by 1
  32. 2025Disentangling Exploration of Large Language Models by Optimal Exploitationcited by 1
  33. 2025Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairnesscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for MLLMscited by 7
  2. 2025When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Trainingcited by 3
  3. 2026Bandit Learning in General Open Multi-agent Systemscited by 0
  4. 2026CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLMcited by 0
  5. 2026Dynamic Governance of Multi-LLM Agent Systems for Collaborative Conversational Outcomescited by 0
  6. 2026Structure, Not Belief: Correlated Thompson Sampling from LLM-Derived Covariance in Combinatorial Semi-Banditscited by 0
  7. 2026PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archivescited by 0