mlsanityData to 14 Oct 2026

Controlled Text Generation with Hidden Representation Transformations

arXiv 2305.19230 ↗First posted 30 May 2023cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose CHRT (Control Hidden Representation Transformation) - a controlled language generation framework that steers large language models to generate text pertaining to certain attributes (such as toxicity). CHRT gains attribute control by modifying the hidden representation of the base model through learned transformations. We employ a contrastive-learning framework to learn these transformations that can be combined to gain multi-attribute control. The effectiveness of CHRT is experimentally shown by comparing it with seven baselines over three attributes. CHRT outperforms all the baselines in the task of detoxification, positive sentiment steering, and text simplification while minimizing the loss in linguistic qualities. Further, our approach has the lowest inference latency of only 0.01 seconds more than the base model, making it the most suitable for high-performance production environments. We open-source our code and release two novel datasets to further propel controlled language generation research.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

33 of 54 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
21 more
  1. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  2. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  3. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  4. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  5. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  6. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  7. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  8. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  9. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 276
  10. 2016Exploring the Limits of Language Modelingcited by 173
  11. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  12. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  13. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  14. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  15. 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
  16. 2016Controlling Output Length in Neural Encoder-Decoderscited by 75
  17. 2020ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learningcited by 72
  18. 2015Towards Universal Paraphrastic Sentence Embeddingscited by 60
  19. 2021Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classificationcited by 43
  20. 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
  21. 2019Contrastive Attention Mechanism for Abstractive Sentence Summarizationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practicescited by 20
  2. 2024Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Modelscited by 1
  3. 2025Towards Lightweight, Adaptive and Attribute-Aware Multi-Aspect Controllable Text Generation with Large Language Modelscited by 0
  4. 2026The Illusion of Control: Why Bare Classifier Inversion Silently Fails in Concept-Bottleneck Text Generationcited by 0