mlsanityData to 7 Oct 2026

DIRECTOR: Generator-Classifiers For Supervised Language Modeling

arXiv 2206.07694 ↗First posted 15 Jun 2022cs.CLcited by 38

Not on the map. Why?

Abstract (arXiv)

Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions. The standard language modeling setup fails to address these issues. In this paper, we introduce a new architecture, sc Director, that consists of a unified generator-classifier with both a language modeling and a classification head for each output token. Training is conducted jointly using both standard language modeling data, and data labeled with desirable and undesirable sequences. Experiments in several settings show that the model has competitive training and decoding speed compared to standard language models while yielding superior results, alleviating known issues while maintaining generation quality. It also outperforms existing model guiding approaches in terms of both accuracy and efficiency.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

28 of 42 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  9. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  10. 2020Recipes for building an open-domain chatbotcited by 593
  11. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  12. 2020Towards a Human-like Open-Domain Chatbotcited by 467
16 more
  1. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  2. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  3. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 315
  4. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  5. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  6. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  7. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  8. 2021Challenges in Detoxifying Language Modelscited by 169
  9. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  10. 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 130
  11. 2020I like fish, especially dolphins: Addressing Contradictions in Dialogue Modelingcited by 68
  12. 2020Controlling Style in Generated Dialoguecited by 38
  13. 2022CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generationcited by 29
  14. 2021Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identitycited by 22
  15. 2022A Simple Contrastive Learning Objective for Alleviating Neural Text Degenerationcited by 18
  16. 2022Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONEScited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 16; 2024: 6; 2025: 4; 2026: 4.

38 papers.

Sort:

By area:

  1. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  2. 2023Controlled Decoding from Language Modelscited by 133
  3. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
  4. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  5. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
  6. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  7. 2022Critic-Guided Decoding for Controlled Text Generationcited by 45
  8. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 43
  9. 2023Using In-Context Learning to Improve Dialogue Safetycited by 41
  10. 2023Click: Controllable Text Generation with Sequence Likelihood Contrastive Learningcited by 41
  11. 2022The CRINGE Loss: Learning what language not to modelcited by 39
  12. 2022Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedbackcited by 35
26 more
  1. 2022DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generationcited by 30
  2. 2022When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labelscited by 20
  3. 2024Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrastcited by 19
  4. 2024Controlled Text Generation for Large Language Model with Dynamic Attribute Graphscited by 16
  5. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  6. 2024RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Frameworkcited by 11
  7. 2023Improving Open Language Models by Learning from Organic Interactionscited by 10
  8. 2022KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learningcited by 9
  9. 2025Decoupling Task-Solving and Output Formatting in LLM Generationcited by 8
  10. 2023Learn What NOT to Learn: Towards Generative Safety in Chatbotscited by 7
  11. 2023Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyondcited by 7
  12. 2023System-Level Natural Language Feedbackcited by 4
  13. 2022DialGuide: Aligning Dialogue Model Behavior with Developer Guidelinescited by 3
  14. 2025ECO Decoding: Entropy-Based Control for Controllability and Fluency in Controllable Dialogue Generationcited by 3
  15. 2026Controlling Reading Ease with Gaze-Guided Text Generationcited by 3
  16. 2023Semantic Space Grounded Weighted Decoding for Multi-Attribute Controllable Dialogue Generationcited by 2
  17. 2023Successor Features for Efficient Multisubject Controlled Text Generationcited by 2
  18. 2023Controlled Text Generation for Black-box Language Models via Score-based Progressive Editorcited by 2
  19. 2024LiFi: Lightweight Controlled Text Generation with Fine-Grained Control Codescited by 2
  20. 2026Test-Time Detoxification without Training or Learning Anythingcited by 2
  21. 2026Mitigating Multimodal Hallucination via Phase-wise Self-rewardcited by 2
  22. 2025LaPA2: Length-Aware Prefix and Prompt Attention Augmentation for Long-Form Controllable Text Generationcited by 1
  23. 2023LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modelingcited by 0
  24. 2023Learning to love diligent trolls: Accounting for rater effects in the dialogue safety taskcited by 0
  25. 2024Enhancing AI Assisted Writing with One-Shot Implicit Negative Feedbackcited by 0
  26. 2026Conditional Attribute Estimation with Autoregressive Sequence Modelscited by 0