mlsanityData to 7 Oct 2026

A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Bandits

arXiv 2601.12945 ↗First posted 19 Jan 2026cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have become powerful and widely used systems for language understanding and generation, while multi-armed bandit (MAB) algorithms provide a principled framework for adaptive decision-making under uncertainty. This survey explores the potential at the intersection of these two fields. As we know, it is the first survey to systematically review the bidirectional interaction between large language models and multi-armed bandits at the component level. We highlight the bidirectional benefits: MAB algorithms address critical LLM challenges, spanning from pre-training to retrieval-augmented generation (RAG) and personalization. Conversely, LLMs enhance MAB systems by redefining core components such as arm definition and environment modeling, thereby improving decision-making in sequential tasks. We analyze existing LLM-enhanced bandit systems and bandit-enhanced LLM systems, providing insights into their design, methodologies, and performance. Key challenges and representative findings are identified to help guide future research. An accompanying GitHub repository that indexes relevant literature is available at https://github.com/bucky1119/Awesome-LLM-Bandit-Interaction.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 109 of its 159 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

109 of 159 references.

References · 109

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
38 more
  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023A Comprehensive Overview of Large Language Modelscited by 639
  3. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  4. 2024Large Language Models: A Surveycited by 506
  5. 2019Introduction to Multi-Armed Banditscited by 475
  6. 2024Tool Learning with Large Language Models: A Surveycited by 221
  7. 2023Nash Learning from Human Feedbackcited by 220
  8. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  9. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  10. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 184
  11. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  12. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  13. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  14. 2023Large Language Models as Evolutionary Optimizerscited by 122
  15. 2025Agentic Large Language Models, a surveycited by 105
  16. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  17. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
  18. 2024Human Alignment of Large Language Models through Online Preference Optimisationcited by 90
  19. 2023Efficient Online Data Mixing For Language Model Pre-Trainingcited by 77
  20. 2025AgentEvolver: Towards Efficient Self-Evolving Agent Systemcited by 77
  21. 2024Can large language models explore in-context?cited by 75
  22. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
  23. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  24. 2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
  25. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 50
  26. 2024Efficient Exploration for LLMscited by 44
  27. 2023Large Language Model-Aware In-Context Learning for Code Generationcited by 42
  28. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 41
  29. 2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitionscited by 38
  30. 2024Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Treescited by 38
  31. 2024Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplarscited by 37
  32. 2024Cost-Effective Online Multi-LLM Selection with Versatile Reward Modelscited by 36
  33. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 36
  34. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 35
  35. 2024Prompt Optimization with Human Feedbackcited by 35
  36. 2023Towards Optimizing with Large Language Modelscited by 34
  37. 2024Harnessing Diversity for Important Data Selection in Pretraining Large Language Modelscited by 34
  38. 2024Do LLM Agents Have Regret? A Case Study in Online Learning and Gamescited by 33

50 most cited of 109 All 109 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning to Orchestrate Agents under Uncertaintycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Should You Use Your Large Language Model to Explore or Exploit?
  2. 2024EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration
  3. 2026When Do We Need LLMs? A Diagnostic for Language-Driven Bandits
  4. 2025Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness
  5. 2024LASeR: Learning to Adaptively Select Reward Models with Multi-Armed Bandits
  6. 2026Cost-Aware Best-LLM Identification using Dueling Feedback
  7. 2025A Multi-Agent Conversational Bandit Approach to Online Evaluation and Selection of User-Aligned LLM Responses
  8. 2026DORA Explorer: Improving the Exploration Ability of LLMs Without Training