# MindFlow+: A Self-Evolving Agent for E-Commerce Customer Service

arXiv: https://arxiv.org/abs/2507.18884 · HTML: https://mlsanity.com/paper/2507.18884/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 29 of 29)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning](https://mlsanity.com/paper/2411.02337/) (2024)
- [A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems](https://mlsanity.com/paper/2203.01387/) (2022)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [RvS: What is Essential for Offline RL via Supervised Learning?](https://mlsanity.com/paper/2112.10751/) (2021)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning](https://mlsanity.com/paper/2205.00445/) (2022)
- [Natural Language Processing: State of The Art, Current Trends and Challenges](https://mlsanity.com/paper/1708.05148/) (2017)
- [Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering](https://mlsanity.com/paper/2404.17723/) (2024)
- [Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations](https://mlsanity.com/paper/2310.05421/) (2023)
- [From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2401.02777/) (2024)
- [CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs](https://mlsanity.com/paper/2404.01343/) (2024)
- [Meaningful Answer Generation of E-Commerce Question-Answering](https://mlsanity.com/paper/2011.07307/) (2020)
- [A survey on fairness of large language models in e-commerce: progress, application, and challenge](https://mlsanity.com/paper/2405.13025/) (2024)
- [Language-Conditioned Offline RL for Multi-Robot Navigation](https://mlsanity.com/paper/2407.20164/) (2024)
- [Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts](https://mlsanity.com/paper/2404.09248/) (2024)
- [Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intents](https://mlsanity.com/paper/2406.11047/) (2024)
- [Natural Language Processing in Customer Service: A Systematic Review](https://mlsanity.com/paper/2212.09523/) (2022)
- [Distantly Supervised Transformers For E-Commerce Product QA](https://mlsanity.com/paper/2104.02947/) (2021)
- [LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble](https://mlsanity.com/paper/2411.17135/) (2024)

## Abstract (arXiv)

High-quality dialogue is crucial for e-commerce customer service, yet traditional intent-based systems struggle with dynamic, multi-turn interactions. We present MindFlow+, a self-evolving dialogue agent that learns domain-specific behavior by combining large language models (LLMs) with imitation learning and offline reinforcement learning (RL). MindFlow+ introduces two data-centric mechanisms to guide learning: tool-augmented demonstration construction, which exposes the model to knowledge-enhanced and agentic (ReAct-style) interactions for effective tool use; and reward-conditioned data modeling, which aligns responses with task-specific goals using reward signals. To evaluate the model's role in response generation, we introduce the AI Contribution Ratio, a novel metric quantifying AI involvement in dialogue. Experiments on real-world e-commerce conversations show that MindFlow+ outperforms strong baselines in contextual relevance, flexibility, and task accuracy. These results demonstrate the potential of combining LLMs tool reasoning, and reward-guided learning to build domain-specialized, context-aware dialogue systems.