mlsanityData to 7 Oct 2026

BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

arXiv 2106.06169 ↗First posted 11 Jun 2021cs.CL cs.AIcited by 75

Not on the map. Why?

Abstract (arXiv)

Maintaining consistent personas is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated persona-dense data are still barriers towards training robust and consistent persona-based dialogue models. In this work, we show how the challenges can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERT-over-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understanding. In particular, to learn the ability of consistency understanding from large-scale non-dialogue inference data, we train the second decoder in an unlikelihood manner. Under different limited data settings, both automatic and human evaluations demonstrate that the proposed model outperforms strong baselines in response quality and persona consistency.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 11 Jun 2021.

Builds on

31 of 40 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  10. 2018Disentangling by Factorisingcited by 614
  11. 2020Recipes for building an open-domain chatbotcited by 593
  12. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
19 more
  1. 2018Towards a Definition of Disentangled Representationscited by 294
  2. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  3. 2016A Persona-Based Neural Conversation Modelcited by 221
  4. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  5. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  6. 2018Dialogue Natural Language Inferencecited by 135
  7. 2018Training Millions of Personalized Dialogue Agentscited by 126
  8. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
  9. 2020You Impress Me: Dialogue Generation via Mutual Persona Perceptioncited by 85
  10. 2019Personalizing Dialogue Agents via Meta-Learningcited by 83
  11. 2019A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Datacited by 74
  12. 2019Personalized Dialogue Generation with Diversified Traitscited by 73
  13. 2019Exploiting Persona Information for Diverse Generation of Conversational Responsescited by 58
  14. 2020Low-Resource Knowledge-Grounded Dialogue Generationcited by 53
  15. 2020Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generationcited by 41
  16. 2020The Adapter-Bot: All-In-One Controllable Conversational Modelcited by 38
  17. 2020XPersona: Evaluating Multilingual Personalized Chatbotcited by 36
  18. 2020Profile Consistency Identification for Open-domain Dialogue Agentscited by 22
  19. 2019Learning to Customize Model Structures for Few-shot Dialogue Generation Taskscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 12; 2023: 23; 2024: 23; 2025: 9; 2026: 3.

75 papers.

Sort:

By area:

  1. 2023Tool Learning with Foundation Modelscited by 302
  2. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  3. 2024Can LLM be a Personalized Judge?cited by 99
  4. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  5. 2023Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMscited by 62
  6. 2024The Oscars of AI Theater: A Survey on Role-Playing with Language Modelscited by 47
  7. 2024UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systemscited by 38
  8. 2024Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphscited by 38
  9. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  10. 2023Writer-Defined AI Personas for On-Demand Feedback Generationcited by 36
  11. 2024Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thoughtcited by 34
  12. 2022A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generationcited by 25
38 more
  1. 2023RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generationcited by 25
  2. 2022Personalized Dialogue Generation with Persona-Adaptive Attentioncited by 24
  3. 2023Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Personacited by 24
  4. 2022COSPLAY: Concept Set Guided Personalized Dialogue Generation Across Both Party Personascited by 20
  5. 2024Recent Trends in Personalized Dialogue Generation: A Review of Datasets, Methodologies, and Evaluationscited by 19
  6. 2024Learning Retrieval Augmentation for Personalized Dialogue Generationcited by 18
  7. 2022Improving Personality Consistency in Conversation by Persona Extendingcited by 17
  8. 2023Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialoguescited by 17
  9. 2023Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularizationcited by 17
  10. 2022There Are a Thousand Hamlets in a Thousand People's Eyes: Enhancing Knowledge-grounded Dialogue with Personal Memorycited by 16
  11. 2023PRODIGy: a PROfile-based DIalogue Generation datasetcited by 16
  12. 2024Selective Prompting Tuning for Personalized Conversations with LLMscited by 15
  13. 2021Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehensioncited by 13
  14. 2024ControlCap: Controllable Region-level Captioningcited by 13
  15. 2024MPCODER: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learningcited by 12
  16. 2023SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluationcited by 11
  17. 2023MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Controlcited by 11
  18. 2023Uncertainty-Aware Unlikelihood Learning Improves Generative Aspect Sentiment Quad Predictioncited by 10
  19. 2024"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learningcited by 10
  20. 2022Building a Personalized Dialogue System with Prompt-Tuningcited by 9
  21. 2022Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systemscited by 9
  22. 2023Towards Personalized Review Summarization by Modeling Historical Reviews from Customer and Product Separatelycited by 9
  23. 2021Detecting Speaker Personas from Conversational Textscited by 8
  24. 2022MulZDG: Multilingual Code-Switching Framework for Zero-shot Dialogue Generationcited by 7
  25. 2024Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMscited by 7
  26. 2024Self-Emotion Blended Dialogue Generation in Social Simulation Agentscited by 7
  27. 2022PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanismcited by 6
  28. 2023UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learningcited by 6
  29. 2023Large Language Models as Source Planner for Personalized Knowledge-grounded Dialoguecited by 6
  30. 2021TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue Systemcited by 5
  31. 2025Role-Playing Evaluation for Large Language Modelscited by 5
  32. 2024MORPHEUS: Modeling Role from Personalized Dialogue History by Exploring and Utilizing Latent Spacecited by 4
  33. 2025Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMscited by 4
  34. 2023ReSee: Responding through Seeing Fine-grained Visual Knowledge in Open-domain Dialoguecited by 3
  35. 2024Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Modelscited by 3
  36. 2024Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlightscited by 3
  37. 2024User-Specific Dialogue Generation with User Profile-Aware Pre-Training Model and Parameter-Efficient Fine-Tuningcited by 3
  38. 2024MCPDial: A Minecraft Persona-driven Dialogue Datasetcited by 3

50 most cited of 75 All 75 (JSON)