Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Recent advances in large language models (LLMs) have enabled fluent dialogue systems, but most remain reactive and struggle in emotionally rich, goal-oriented settings such as marketing conversations. To address this limitation, we propose AffectMind, a multimodal affective dialogue agent that performs proactive reasoning and dynamic knowledge grounding to sustain emotionally aligned and persuasive interactions. AffectMind combines three components: a Proactive Knowledge Grounding Network (PKGN) that continuously updates factual and affective context from text, vision, and prosody; an Emotion--Intent Alignment Model (EIAM) that jointly models user emotion and purchase intent to adapt persuasion strategies; and a Reinforced Discourse Loop (RDL) that optimizes emotional coherence and engagement via reinforcement signals from user responses. Experiments on two newly curated marketing dialogue datasets, MM-ConvMarket and AffectPromo, show that AffectMind outperforms strong LLM-based baselines in emotional consistency (+26%), persuasive success rate (+19%), and long-term user engagement (+23%), highlighting emotion-grounded proactivity as a key capability for commercial multimodal agents.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 18 of its 48 references that are in the map. First posted on arXiv 21 Nov 2025.
Builds on
18 of 48 references.
References · 18
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2015VQA: Visual Question Answeringcited by 2,489
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
- 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
- 2016Visual Dialogcited by 377
- 2017Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memorycited by 167
- 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
- 2020MIME: MIMicking Emotions for Empathetic Response Generationcited by 114
6 more
- 2019Proactive Human-Machine Conversation with Explicit Conversation Goalscited by 87
- 2018Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systemscited by 70
- 2019Target-Guided Open-Domain Conversationcited by 66
- 2024Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM argumentscited by 30
- 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Taskscited by 27
- 2020Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstrationcited by 20
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026EVLA: An Electro-Aware Multimodal Assistant for Physically-Grounded Driving Reasoning and Controlcited by 0
- 2026A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctionscited by 0
- 2026Beyond Skepticism: Evaluating LLMs Pedagogical Intent Reasoning with the Adaptive Pedagogical Vigilance Frameworkcited by 0
- 2026FedCausal-Dyn: A Causal-Dynamic Paradigm for Federated Learning under Dynamic Feature Driftcited by 0
- 2026Beyond Skepticism: Evaluating LLMs Pedagogical Intent Reasoning with the Adaptive Pedagogical Vigilance Framework2 Jul 2026
- 2026A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions27 Jun 2026
- 2026EVLA: An Electro-Aware Multimodal Assistant for Physically-Grounded Driving Reasoning and Control27 Jun 2026
- 2026FedCausal-Dyn: A Causal-Dynamic Paradigm for Federated Learning under Dynamic Feature Drift20 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2023Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversations
- 2026User-Aware Active Knowledge Acquisition for Emotional Support Dialogue
- 2026TEA-Bench: A Systematic Benchmarking of Tool-enhanced Emotional Support Dialogue Agent
- 2026Unlocking Proactivity in Task-Oriented Dialogue
- 2026From Triggers to Emotions: A CPM-Grounded Appraisal Multi-Agent for Dynamic Emotional Evolution in Persona-Based Dialogue
- 2026Reading the Mood Behind Words: Integrating Prosody-Derived Emotional Context into Socially Responsive VR Agents
- 2024The Future of Cognitive Strategy-enhanced Persuasive Dialogue Agents: New Perspectives and Trends
- 2026AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition