# A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing

arXiv: https://arxiv.org/abs/2506.06316 · HTML: https://mlsanity.com/paper/2506.06316/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 7 of 7)

- [Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video](https://mlsanity.com/paper/2203.06667/) (2022)
- [More but Correct: Generating Diversified and Entity-revised Medical Response](https://mlsanity.com/paper/2108.01266/) (2021)
- [LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments](https://mlsanity.com/paper/2406.02611/) (2024)
- [Towards Better Chinese-centric Neural Machine Translation for Low-resource Languages](https://mlsanity.com/paper/2204.04344/) (2022)
- [Psychological Health Knowledge-Enhanced LLM-based Social Network Crisis Intervention Text Transfer Recognition Method](https://mlsanity.com/paper/2504.07983/) (2025)
- [Research on Personalized Medical Intervention Strategy Generation System based on Group Relative Policy Optimization and Time-Series Data Fusion](https://mlsanity.com/paper/2504.18631/) (2025)
- [A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing](https://mlsanity.com/paper/2308.09790/) (2023)

## Built on by (most cited first, 1 of 1)

- [Multimodal Health Risk Prediction System for Chronic Diseases via Vision-Language Fusion and Large Language Models](https://mlsanity.com/paper/2509.18221/) (2025)

## Abstract (arXiv)

For personalized marketing, a new challenge of how to effectively algorithm the A/B testing to maximize user response is urgently to be overcome. In this paper, we present a new approach, the RL-LLM-AB test framework, for using reinforcement learning strategy optimization combined with LLM to automate and personalize A/B tests. The RL-LLM-AB test is built upon the pre-trained instruction-tuned language model. It first generates A/B versions of candidate content variants using a Prompt-Conditioned Generator, and then dynamically embeds and fuses the user portrait and the context of the current query with the multi-modal perception module to constitute the current interaction state. The content version is then selected in real-time through the policy optimization module with an Actor-Critic structure, and long-term revenue is estimated according to real-time feedback (such as click-through rate and conversion rate). Furthermore, a Memory-Augmented Reward Estimator is embedded into the framework to capture long-term user preference drift, which helps to generalize policy across multiple users and content contexts. Numerical results demonstrate the superiority of our proposed RL-LLM-ABTest over existing A/B testing methods, including classical A/B testing, Contextual Bandits, and benchmark reinforcement learning approaches on real-world marketing data.