# Neural Response Ranking for Social Conversation: A Data-Efficient Approach

arXiv: https://arxiv.org/abs/1811.00967 · HTML: https://mlsanity.com/paper/1811.00967/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2018.

## Built on by (most cited first, 9 of 9)

- [Leveraging Implicit Feedback from Deployment Data in Dialogue](https://mlsanity.com/paper/2307.14117/) (2023)
- [Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialogue](https://mlsanity.com/paper/2110.08094/) (2021)
- [Modeling Performance in Open-Domain Dialogue with PARADISE](https://mlsanity.com/paper/2110.11164/) (2021)
- [GraphWOZ: Dialogue Management with Conversational Knowledge Graphs](https://mlsanity.com/paper/2211.12852/) (2022)
- [Athena 2.0: Contextualized Dialogue Management for an Alexa Prize SocialBot](https://mlsanity.com/paper/2111.02519/) (2021)
- [A Systematic Evaluation of Response Selection for Open Domain Dialogue](https://mlsanity.com/paper/2208.04379/) (2022)
- [Improving Open-Domain Dialogue Evaluation with a Causal Inference Model](https://mlsanity.com/paper/2301.13372/) (2023)
- [A Transformer-based Response Evaluator for Open-Domain Spoken Conversation](https://mlsanity.com/paper/2302.04424/) (2023)
- [Athena 2.0: Discourse and User Modeling in Open Domain Dialogue](https://mlsanity.com/paper/2308.01887/) (2023)

## Abstract (arXiv)

The overall objective of 'social' dialogue systems is to support engaging, entertaining, and lengthy conversations on a wide variety of topics, including social chit-chat. Apart from raw dialogue data, user-provided ratings are the most common signal used to train such systems to produce engaging responses. In this paper we show that social dialogue systems can be trained effectively from raw unannotated data. Using a dataset of real conversations collected in the 2017 Alexa Prize challenge, we developed a neural ranker for selecting 'good' system responses to user utterances, i.e. responses which are likely to lead to long and engaging conversations. We show that (1) our neural ranker consistently outperforms several strong baselines when trained to optimise for user ratings; (2) when trained on larger amounts of data and only using conversation length as the objective, the ranker performs better than the one trained using ratings -- ultimately reaching a Precision@1 of 0.87. This advance will make data collection for social conversational agents simpler and less expensive in the future.