mlsanityData to 7 Oct 2026

Learning to Generate Reviews and Discovering Sentiment

arXiv 1704.01444 ↗First posted 5 Apr 2017cs.LG cs.CL cs.NEcited by 147

Not on the map. Why?

Abstract (arXiv)

We explore the properties of byte-level recurrent language models. When given sufficient amounts of capacity, training data, and compute time, the representations learned by these models include disentangled features corresponding to high-level concepts. Specifically, we find a single unit which performs sentiment analysis. These representations, learned in an unsupervised manner, achieve state of the art on the binary subset of the Stanford Sentiment Treebank. They are also very data efficient. When using only a handful of labeled examples, our approach matches the performance of strong baselines trained on full datasets. We also demonstrate the sentiment unit has a direct influence on the generative process of the model. Simply fixing its value to be positive or negative generates samples with the corresponding positive or negative sentiment.

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 15; 2022: 23; 2023: 35; 2024: 35; 2025: 20; 2026: 18.

147 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  8. 2024The Platonic Representation Hypothesiscited by 362
  9. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  10. 2023Challenges and Applications of Large Language Modelscited by 251
  11. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  12. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
38 more
  1. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  2. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  3. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  4. 2022Language Models as Agent Modelscited by 131
  5. 2023The Science of Detecting LLM-Generated Textscited by 126
  6. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 124
  7. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  8. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  9. 2021ArtEmis: Affective Language for Visual Artcited by 103
  10. 2024Universal Neurons in GPT2 Language Modelscited by 100
  11. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  12. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  13. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  14. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  15. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  16. 2025Persona Features Control Emergent Misalignmentcited by 76
  17. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
  18. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  19. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  20. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 60
  21. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  22. 2024Confidence Regulation Neurons in Language Modelscited by 57
  23. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 51
  24. 2024Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approachcited by 48
  25. 2023Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuningcited by 45
  26. 2021The Low-Dimensional Linear Geometry of Contextualized Word Representationscited by 39
  27. 2023Emergent Modularity in Pre-trained Transformerscited by 37
  28. 2024Interpreting the Second-Order Effects of Neurons in CLIPcited by 36
  29. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  30. 2023A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Materialcited by 34
  31. 2023A Preliminary Study of the Intrinsic Relationship between Complexity and Alignmentcited by 33
  32. 2024Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Modelscited by 31
  33. 2021Learning to Generate Music With Sentimentcited by 29
  34. 2023Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarkscited by 29
  35. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  36. 2021Attribute Alignment: Controlling Text Generation from Pre-trained Language Modelscited by 27
  37. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 26
  38. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26

50 most cited of 147 All 147 (JSON)