mlsanityData to 14 Oct 2026

Towards Reliable Multi-Agent Systems for Marketing Applications via Reflection, Memory, and Planning

arXiv 2508.11120 ↗First posted 14 Aug 2025cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Recent advances in large language models (LLMs) enabled the development of AI agents that can plan and interact with tools to complete complex tasks. However, literature on their reliability in real-world applications remains limited. In this paper, we introduce a multi-agent framework for a marketing task: audience curation. To solve this, we introduce a framework called RAMP that iteratively plans, calls tools, verifies the output, and generates suggestions to improve the quality of the audience generated. Additionally, we equip the model with a long-term memory store, which is a knowledge base of client-specific facts and past queries. Overall, we demonstrate the use of LLM planning and memory, which increases accuracy by 28 percentage points on a set of 88 evaluation queries. Moreover, we show the impact of iterative verification and reflection on more ambiguous queries, showing progressively better recall (roughly +20 percentage points) with more verify/reflect iterations on a smaller challenge set, and higher user satisfaction. Our results provide practical insights for deploying reliable LLM-based systems in dynamic, industry-facing environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 21 of its 34 references that are in the map. First posted on arXiv 14 Aug 2025.

Builds on

21 of 34 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Towards Autonomous Agents: Adaptive-planning, Reasoning, and Acting in Language Models12 Aug
  2. 2024ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents28 Jun
  3. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey17 Apr
  4. 2024Understanding the planning of LLM agents: A survey5 Feb
  5. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents1 Feb
  6. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage7 Aug

Other references · 15

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  9. 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 394
  10. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  11. 2024Learning to Use Tools via Cooperative and Interactive Agentscited by 54
  12. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
3 more
  1. 2024MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Codecited by 39
  2. 2025Neuro-Symbolic AI in 2024: A Systematic Reviewcited by 38
  3. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 36

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MAG-V: A Multi-Agent Framework for Synthetic Data Generation and Verification
  2. 2026AD-Bench: A Real-World, Trajectory-Aware Advertising Analytics Benchmark for LLM Agents
  3. 2025Enabling Personalized Long-term Interactions in LLM-based Agents through Persistent Memory and User Profiles
  4. 2026EComStage: Stage-wise and Orientation-specific Benchmarking for Large Language Models in E-commerce
  5. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  6. 2025Parallelism Meets Adaptiveness: Scalable Documents Understanding in Multi-Agent LLM Systems
  7. 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration
  8. 2025Agentic Large Language Models, a survey