# CheerBots: Chatbots toward Empathy and Emotionusing Reinforcement Learning

arXiv: https://arxiv.org/abs/2110.03949 · HTML: https://mlsanity.com/paper/2110.03949/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [Pathologies of Neural Models Make Interpretations Difficult](https://mlsanity.com/paper/1804.07781/) (2018)
- [Neural Responding Machine for Short-Text Conversation](https://mlsanity.com/paper/1503.02364/) (2015)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Polite Dialogue Generation Without Parallel Data](https://mlsanity.com/paper/1805.03162/) (2018)
- [Learning Semantic Textual Similarity from Conversations](https://mlsanity.com/paper/1804.07754/) (2018)
- [Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation](https://mlsanity.com/paper/1905.12926/) (2019)
- [CAiRE: An Empathetic Neural Chatbot](https://mlsanity.com/paper/1907.12108/) (2019)
- [A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots](https://mlsanity.com/paper/1710.11344/) (2017)
- [Production Ready Chatbots: Generate if not Retrieve](https://mlsanity.com/paper/1711.09684/) (2017)

## Built on by (most cited first, 7 of 7)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025)
- [Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities](https://mlsanity.com/paper/2206.05017/) (2022)
- [Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks](https://mlsanity.com/paper/2210.05664/) (2022)
- [Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations](https://mlsanity.com/paper/2401.04883/) (2024)
- [Mitigating Toxic Degeneration with Empathetic Data: Exploring the Relationship Between Toxicity and Empathy](https://mlsanity.com/paper/2205.07233/) (2022)
- [EmoLLM: Appraisal-Grounded Cognitive-Emotional Co-Reasoning in Large Language Models](https://mlsanity.com/paper/2603.16553/) (2026)

## Abstract (arXiv)

Apart from the coherence and fluency of responses, an empathetic chatbot emphasizes more on people's feelings. By considering altruistic behaviors between human interaction, empathetic chatbots enable people to get a better interactive and supportive experience. This study presents a framework whereby several empathetic chatbots are based on understanding users' implied feelings and replying empathetically for multiple dialogue turns. We call these chatbots CheerBots. CheerBots can be retrieval-based or generative-based and were finetuned by deep reinforcement learning. To respond in an empathetic way, we develop a simulating agent, a Conceptual Human Model, as aids for CheerBots in training with considerations on changes in user's emotional states in the future to arouse sympathy. Finally, automatic metrics and human rating results demonstrate that CheerBots outperform other baseline chatbots and achieves reciprocal altruism. The code and the pre-trained models will be made available.