mlsanityData to 14 Oct 2026

The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends

arXiv 2409.14195 ↗First posted 21 Sep 2024cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

In the era of large language models (LLMs), a vast amount of conversation logs will be accumulated thanks to the rapid development trend of language UI. Conversation Analysis (CA) strives to uncover and analyze critical information from conversation data, streamlining manual processes and supporting business insights and decision-making. The need for CA to extract actionable insights and drive empowerment is becoming increasingly prominent and attracting widespread attention. However, the lack of a clear scope for CA leads to a dispersion of various techniques, making it difficult to form a systematic technical synergy to empower business applications. In this paper, we perform a thorough review and systematize CA task to summarize the existing related work. Specifically, we formally define CA task to confront the fragmented and chaotic landscape in this field, and derive four key steps of CA from conversation scene reconstruction, to in-depth attribution analysis, and then to performing targeted training, finally generating conversations based on the targeted training for achieving the specific goals. In addition, we showcase the relevant benchmarks, discuss potential challenges and point out future directions in both industry and academia. In view of current advancements, it is evident that the majority of efforts are still concentrated on the analysis of shallow conversation elements, which presents a considerable gap between the research and business, and with the assist of LLMs, recent work has shown a trend towards research on causality and strategic tasks which are sophisticated and high-level. The analyzed experiences and insights will inevitably have broader application value in business operations that target conversation logs.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 100 of its 152 references that are in the map. First posted on arXiv 21 Sep 2024.

Builds on

100 of 152 references.

References · 100

  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  3. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  5. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  6. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  7. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  8. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  9. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  10. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  11. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  12. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 434
38 more
  1. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  2. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
  3. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  4. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  5. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  6. 2019Survey on Evaluation Methods for Dialogue Systemscited by 126
  7. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 123
  8. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  9. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  10. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 122
  11. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 116
  12. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 114
  13. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 108
  14. 2020Towards Conversational Recommendation over Multi-Type Dialogscited by 107
  15. 2024Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignmentcited by 104
  16. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 98
  17. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 95
  18. 2020Towards Topic-Guided Conversational Recommender Systemcited by 94
  19. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 94
  20. 2020INSPIRED: Toward Sociable Recommendation Dialog Systemscited by 93
  21. 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agentscited by 93
  22. 2019Proactive Human-Machine Conversation with Explicit Conversation Goalscited by 88
  23. 2022AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversationcited by 79
  24. 2023Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agentscited by 77
  25. 2021Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systemscited by 75
  26. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  27. 2019Target-Guided Open-Domain Conversationcited by 66
  28. 2020Recognizing Emotion Cause in Conversationscited by 66
  29. 2022Keep Me Updated! Memory Management in Long-term Conversationscited by 63
  30. 2022The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMscited by 58
  31. 2024Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluationcited by 58
  32. 2020Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Surveycited by 51
  33. 2024Towards Lifelong Dialogue Agents via Timeline-based Memory Managementcited by 51
  34. 2020Adding Chit-Chat to Enhance Task-Oriented Dialoguescited by 48
  35. 2023Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversationscited by 47
  36. 2023Faithful Persona-based Conversational Dataset Generation with Large Language Modelscited by 47
  37. 2024HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapycited by 47
  38. 2021DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendationcited by 46

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 3.

11 papers.

  1. 2024IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimizationcited by 31
  2. 2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
  3. 2025Adaptive Social Learning via Mode Policy Optimization for Language Agentscited by 15
  4. 2025Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimizationcited by 10
  5. 2025EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Modelscited by 9
  6. 2024DEMO: Reframing Dialogue Interaction with Fine-grained Element Modelingcited by 6
  7. 2025Toxicity Begets Toxicity: Unraveling Conversational Chains in Political Podcastscited by 4
  8. 2025Finding Diamonds in Conversation Haystacks: A Benchmark for Conversational Data Retrievalcited by 0
  9. 2026Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debuggingcited by 0
  10. 2026Can AI Guess What You Know? Performance Comparison of Large Language Models for Human Domain Knowledge Estimation From Communication Logscited by 0
  11. 2026Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewardingcited by 0