Empathic Prompting: Non-Verbal Context Integration for Multimodal LLM Conversations
On the map: Multimodal LLMs / vision-language models → Multimodal emotion understanding
Abstract (arXiv)
We present Empathic Prompting, a novel framework for multimodal human-AI interaction that enriches Large Language Model (LLM) conversations with implicit non-verbal context. The system integrates a commercial facial expression recognition service to capture users' emotional cues and embeds them as contextual signals during prompting. Unlike traditional multimodal interfaces, empathic prompting requires no explicit user control; instead, it unobtrusively augments textual input with affective information for conversational and smoothness alignment. The architecture is modular and scalable, allowing integration of additional non-verbal modules. We describe the system design, implemented through a locally deployed DeepSeek instance, and report a preliminary service and usability evaluation (N=5). Results show consistent integration of non-verbal input into coherent LLM outputs, with participants highlighting conversational fluidity. Beyond this proof of concept, empathic prompting points to applications in chatbot-mediated communication, particularly in domains like healthcare or education, where users' emotional signals are critical yet often opaque in verbal exchanges.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 79 references that are in the map. First posted on arXiv 23 Oct 2025.
Builds on
17 of 79 references.
References · 17
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
- 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
- 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
5 more
- 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 115
- 2023EQ-Bench: An Emotional Intelligence Benchmark for Large Language Modelscited by 75
- 2025Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreementcited by 24
- 2025Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Modelscited by 11
- 2025In-Context Examples Matter: Improving Emotion Recognition in Conversation with Instruction Tuningcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational Agents
- 2025E-THER: A Multimodal Dataset for Empathic AI -- Towards Emotional Mismatch Awareness
- 2025From Measurement to Expertise: Empathetic Expert Adapters for Context-Based Empathy in Conversational AI Agents
- 2026Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathy
- 2024EmPO: Emotion Grounding for Empathetic Response Generation through Preference Optimization
- 2025Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data
- 2026HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs
- 2025HumanSense: From Multimodal Perception to Empathetic Context-Aware Responses through Reasoning MLLMs