mlsanityData to 14 Oct 2026

A Complete Survey on LLM-based AI Chatbots

arXiv 2406.16937 ↗First posted 17 Jun 2024cs.CL cs.AIcited by 71

Not on the map. Why?

Abstract (arXiv)

The past few decades have witnessed an upsurge in data, forming the foundation for data-hungry, learning-based AI technology. Conversational agents, often referred to as AI chatbots, rely heavily on such data to train large language models (LLMs) and generate new content (knowledge) in response to user prompts. With the advent of OpenAI's ChatGPT, LLM-based chatbots have set new standards in the AI community. This paper presents a complete survey of the evolution and deployment of LLM-based chatbots in various sectors. We first summarize the development of foundational chatbots, followed by the evolution of LLMs, and then provide an overview of LLM-based chatbots currently in use and those in the development phase. Recognizing AI chatbots as tools for generating new knowledge, we explore their diverse applications across various industries. We then discuss the open challenges, considering how the data used to train the LLMs and the misuse of the generated knowledge can cause several issues. Finally, we explore the future outlook to augment their efficiency and reliability in numerous applications. By addressing key milestones and the present-day context of LLM-based chatbots, our survey invites readers to delve deeper into this realm, reflecting on how their next generation will reshape conversational AI.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). Builds on 65 of its 207 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

65 of 207 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2020Scaling Laws for Neural Language Modelscited by 6,586
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
38 more
  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2022Emergent Abilities of Large Language Modelscited by 2,512
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  9. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  10. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  11. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  12. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  13. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  14. 2019Green AIcited by 627
  15. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  16. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  17. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  18. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  19. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
  20. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  21. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 246
  22. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  23. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 189
  24. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 185
  25. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  26. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  27. 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
  28. 2023A Categorical Archive of ChatGPT Failurescited by 168
  29. 2022ChatGPT: The End of Online Exam Integrity?cited by 88
  30. 2018From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbotscited by 85
  31. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  32. 2023Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?cited by 54
  33. 2023ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishingcited by 52
  34. 2023Will ChatGPT get you caught? Rethinking of Plagiarism Detectioncited by 41
  35. 2023Beyond the Safeguards: Exploring the Security Risks of ChatGPTcited by 36
  36. 2023LLMs for Science: Usage for Code Generation and Data Analysiscited by 34
  37. 2022Green Learning: Introduction, Examples and Outlookcited by 30
  38. 2023ChatGPT: Applications, Opportunities, and Threatscited by 28

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 48; 2026: 12.

71 papers.

Sort:

By area:

  1. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 249
  2. 2024GUI Agents: A Surveycited by 103
  3. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 78
  4. 2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
  5. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 56
  6. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  7. 2025Measuring Agents in Productioncited by 37
  8. 2025Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducementscited by 32
  9. 2025Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systemscited by 26
  10. 2024TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Studentscited by 25
  11. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  12. 2025LLM-Powered AI Agent Systems and Their Applications in Industrycited by 22
38 more
  1. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 20
  2. 2025SEA-HELM: Southeast Asian Holistic Evaluation of Language Modelscited by 19
  3. 2025Evaluating LLM Metrics Through Real-World Capabilitiescited by 18
  4. 2024Automated test generation to evaluate tool-augmented LLMs as conversational AI agentscited by 17
  5. 2026Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agentscited by 16
  6. 2025Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignmentcited by 14
  7. 2025Position: The Real Barrier to LLM Agent Usability is Agentic ROIcited by 12
  8. 2025Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool usecited by 10
  9. 2025Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoderscited by 9
  10. 2025Evaluating Robustness of Large Language Models Against Multilingual Typographical Errorscited by 9
  11. 2025Meta-Cultural Competence: Climbing the Right Hill of Cultural Awarenesscited by 8
  12. 2025A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personascited by 8
  13. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surfacecited by 8
  14. 2024LLM-DaaS: LLM-driven Drone-as-a-Service Operations from Text User Requestscited by 7
  15. 2025Consistency of Responses and Continuations Generated by Large Language Models on Social Mediacited by 7
  16. 2025HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUscited by 7
  17. 2025A Survey on GUI Agents with Foundation Models Enhanced by Reinforcement Learningcited by 7
  18. 2025Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inferencecited by 6
  19. 2025Hallucinations are inevitable but can be made statistically negligiblecited by 5
  20. 2025Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue Historycited by 5
  21. 2025PoETa v2: Toward More Robust Evaluation of Large Language Models in Portuguesecited by 5
  22. 2025This Is Your Doge, If It Please You: Exploring Deception and Robustness in Mixture of LLMscited by 4
  23. 2025PolyG: Adaptive Graph Traversal for Diverse GraphRAG Questionscited by 4
  24. 2025Sword and Shield: Uses and Strategies of LLMs in Navigating Disinformationcited by 4
  25. 2025HGCA: Hybrid GPU-CPU Attention for Long Context LLM Inferencecited by 4
  26. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAscited by 4
  27. 2026LAPS: A Length-Aware-Prefill LLM Serving Systemcited by 4
  28. 2025The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Modelscited by 3
  29. 2025BanglaTalk: Towards Real-Time Speech Assistance for Bengali Regional Dialectscited by 3
  30. 2025Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systemscited by 2
  31. 2025Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Spacecited by 2
  32. 2025MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inferencecited by 2
  33. 2025Prompting-in-a-Series: Psychology-Informed Contents and Embeddings for Personality Recognition With Decoder-Only Modelscited by 2
  34. 2026COMPASS: A Framework for Evaluating Organization-Specific Policy Alignment in LLMscited by 2
  35. 2025AI Chatbots as Professional Service Agents: Developing a Professional Identitycited by 1
  36. 2025Combating Toxic Language: A Review of LLM-Based Strategies for Software Engineeringcited by 1
  37. 2025Exploring Anthropomorphism in Conversational Agents for Environmental Sustainabilitycited by 1
  38. 2025GOFAI meets Generative AI: Development of Expert Systems by means of Large Language Modelscited by 1

50 most cited of 71 All 71 (JSON)