mlsanityData to 14 Oct 2026

CTRL: A Conditional Transformer Language Model for Controllable Generation

arXiv 1909.05858 ↗First posted 11 Sep 2019cs.CLcited by 762

Not on the map. Why?

Cited by 34 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text. We release CTRL, a 1.63 billion-parameter conditional transformer language model, trained to condition on control codes that govern style, content, and task-specific behavior. Control codes were derived from structure that naturally co-occurs with raw text, preserving the advantages of unsupervised learning while providing more explicit control over text generation. These codes also allow CTRL to predict which parts of the training data are most likely given a sequence. This provides a potential method for analyzing large amounts of data via model-based source attribution. We have released multiple full-sized, pretrained versions of CTRL at https://github.com/salesforce/ctrl.

Key facts. Cited by 762 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 135; 2022: 163; 2023: 175; 2024: 129; 2025: 86; 2026: 74.

762 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  8. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  9. 2023LIMA: Less Is More for Alignmentcited by 1,100
  10. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  11. 2022A Generalist Agentcited by 808
  12. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
38 more
  1. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  2. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  3. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  4. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  5. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  6. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  7. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  8. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  9. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  10. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  11. 2022Teaching language models to support answers with verified quotescited by 292
  12. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  13. 2023Challenges and Applications of Large Language Modelscited by 254
  14. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 254
  15. 2023Pretraining Language Models with Human Preferencescited by 247
  16. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 243
  17. 2023Large Language Model Alignment: A Surveycited by 243
  18. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  19. 2021Hash Layers For Large Sparse Modelscited by 237
  20. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  21. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 229
  22. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 222
  23. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  24. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  25. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  26. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  27. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
  28. 2022Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Modelscited by 191
  29. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  30. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  31. 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 163
  32. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 155
  33. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  34. 2023Likelihood-Based Diffusion Language Modelscited by 151
  35. 2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
  36. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 148
  37. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 148
  38. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147

50 most cited of 762 All 762 (JSON)