mlsanityData to 14 Oct 2026

Domain Private Transformers for Multi-Domain Dialog Systems

arXiv 2305.14208 ↗First posted 23 May 2023cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large, general purpose language models have demonstrated impressive performance across many different conversational domains. While multi-domain language models achieve low overall perplexity, their outputs are not guaranteed to stay within the domain of a given input prompt. This paper proposes domain privacy as a novel way to quantify how likely a conditional language model will leak across domains. We also develop policy functions based on token-level domain classification, and propose an efficient fine-tuning method to improve the trained model's domain privacy. Experiments on membership inference attacks show that our proposed method has comparable resiliency to methods adapted from recent literature on differentially private language models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

18 of 39 references.

References · 18

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  5. 2021Membership Inference Attacks From First Principlescited by 878
  6. 2017Renyi Differential Privacycited by 650
  7. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  8. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  9. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  10. 2020Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learningcited by 139
  11. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
  12. 2020UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2cited by 111
6 more
  1. 2019Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systemscited by 110
  2. 2021Selective Differential Privacy for Language Modelingcited by 62
  3. 2021Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domainscited by 36
  4. 2020Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domainscited by 33
  5. 2022Submix: Practical Private Prediction for Large-Scale Language Modelscited by 31
  6. 2021Private Multi-Task Learning: Formulation and Applications to Federated Learningcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions
  2. 2025A Survey on Privacy Risks and Protection in Large Language Models
  3. 2024Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting
  4. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
  5. 2026Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation
  6. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
  7. 2025DP-Fusion: Token-Level Differentially Private Inference for Large Language Models
  8. 2024On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models