# Generative and Discriminative Text Classification with Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1703.01898 · HTML: https://mlsanity.com/paper/1703.01898/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2017.

## Built on by (most cited first, 30 of 31)

- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [Gradient-Based Constrained Sampling from Language Models](https://mlsanity.com/paper/2205.12558/) (2022)
- [Toxicity Detection with Generative Prompt-based Inference](https://mlsanity.com/paper/2205.12390/) (2022)
- [Multiscale Flow for Robust and Optimal Cosmological Analysis](https://mlsanity.com/paper/2306.04689/) (2023)
- [Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions](https://mlsanity.com/paper/2311.07115/) (2023)
- [Generative or Discriminative? Revisiting Text Classification in the Era of Transformers](https://mlsanity.com/paper/2506.12181/) (2025)
- [Energy-based Unknown Intent Detection with Data Manipulation](https://mlsanity.com/paper/2107.12542/) (2021)
- [Diffusion map particle systems for generative modeling](https://mlsanity.com/paper/2304.00200/) (2023)
- [Zero-Shot Multi-Label Topic Inference with Sentence Encoders](https://mlsanity.com/paper/2304.07382/) (2023)
- [Balanced and Explainable Social Media Analysis for Public Health with Large Language Models](https://mlsanity.com/paper/2309.05951/) (2023)
- [Attention-based Contextual Language Model Adaptation for Speech Recognition](https://mlsanity.com/paper/2106.01451/) (2021)
- [Memory and attention in deep learning](https://mlsanity.com/paper/2107.01390/) (2021)
- [Discriminative and Generative Transformer-based Models For Situation Entity Classification](https://mlsanity.com/paper/2109.07434/) (2021)
- [Forming Trees with Treeformers](https://mlsanity.com/paper/2207.06960/) (2022)
- [SLCNN: Sentence-Level Convolutional Neural Network for Text Classification](https://mlsanity.com/paper/2301.11696/) (2023)
- [Stable generative modeling using Schr"odinger bridges](https://mlsanity.com/paper/2401.04372/) (2024)
- [Which Discriminator for Cooperative Text Generation?](https://mlsanity.com/paper/2204.11586/) (2022)
- [Conformal Predictor for Improving Zero-shot Text Classification Efficiency](https://mlsanity.com/paper/2210.12619/) (2022)
- [Small Language Models in the Real World: Insights from Industrial Text Classification](https://mlsanity.com/paper/2505.16078/) (2025)
- [Generative and Discriminative Deep Belief Network Classifiers: Comparisons Under an Approximate Computing Framework](https://mlsanity.com/paper/2102.00534/) (2021)
- [The Hidden Cost of Modeling P(X): Vulnerability to Membership Inference Attacks in Generative Text Classifiers](https://mlsanity.com/paper/2510.16122/) (2025)
- [Maps Search Misspelling Detection Leveraging Domain-Augmented Contextual Representations](https://mlsanity.com/paper/2108.06842/) (2021)
- [Bypassing Logits Bias in Online Class-Incremental Learning with a Generative Framework](https://mlsanity.com/paper/2205.09347/) (2022)
- [Contrastive Graph Multimodal Model for Text Classification in Videos](https://mlsanity.com/paper/2206.02343/) (2022)
- [CapsF: Capsule Fusion for Extracting psychiatric stressors for suicide from twitter](https://mlsanity.com/paper/2403.15391/) (2024)
- [Post-Training Quantization of Generative and Discriminative LSTM Text Classifiers: A Study of Calibration, Class Balance, and Robustness](https://mlsanity.com/paper/2507.09687/) (2025)
- [Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder](https://mlsanity.com/paper/2603.05528/) (2026)
- [Enhancing Science Classroom Discourse Analysis through Joint Multi-Task Learning for Reasoning-Component Classification](https://mlsanity.com/paper/2604.21137/) (2026)
- [Generative Long-term User Interest Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/2605.15905/) (2026)
- [Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails](https://mlsanity.com/paper/2609.33634/) (2026)

## Abstract (arXiv)

We empirically characterize the performance of discriminative and generative LSTM models for text classification. We find that although RNN-based generative models are more powerful than their bag-of-words ancestors (e.g., they account for conditional dependencies across words in a document), they have higher asymptotic error rates than discriminatively trained RNN models. However we also find that generative models approach their asymptotic error rate more rapidly than their discriminative counterparts---the same pattern that Ng & Jordan (2001) proved holds for linear classification models that make more naive conditional independence assumptions. Building on this finding, we hypothesize that RNN-based generative classification models will be more robust to shifts in the data distribution. This hypothesis is confirmed in a series of experiments in zero-shot and continual learning settings that show that generative models substantially outperform discriminative models.