mlsanityData to 7 Oct 2026

A Hybrid of Generative and Discriminative Models Based on the Gaussian-coupled Softmax Layer

arXiv 2305.05912 ↗First posted 10 May 2023cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Generative models have advantageous characteristics for classification tasks such as the availability of unsupervised data and calibrated confidence, whereas discriminative models have advantages in terms of the simplicity of their model structures and learning algorithms and their ability to outperform their generative counterparts. In this paper, we propose a method to train a hybrid of discriminative and generative models in a single neural network (NN), which exhibits the characteristics of both models. The key idea is the Gaussian-coupled softmax layer, which is a fully connected layer with a softmax activation function coupled with Gaussian distributions. This layer can be embedded into an NN-based classifier and allows the classifier to estimate both the class posterior distribution and the class-conditional data distribution. We demonstrate that the proposed hybrid model can be applied to semi-supervised learning and confidence calibration.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

25 of 48 references.

References · 25

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2016Wide Residual Networkscited by 3,057
  4. 2015Variational Inference with Normalizing Flowscited by 2,161
  5. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  6. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  7. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  8. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  9. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  10. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  11. 2020Meta Pseudo Labelscited by 372
  12. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
13 more
  1. 2019Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentationcited by 162
  2. 2018Deep Co-Training for Semi-Supervised Image Recognitioncited by 145
  3. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 127
  4. 2020Multivariate Confidence Calibration for Object Detectioncited by 98
  5. 2020Residual Energy-Based Models for Text Generationcited by 95
  6. 2021Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibrationcited by 91
  7. 2022BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotationcited by 54
  8. 2023Rethinking Confidence Calibration for Failure Predictioncited by 43
  9. 2018Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Modelcited by 34
  10. 2018Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferencescited by 25
  11. 2016Infinite Variational Autoencoder for Semi-Supervised Learningcited by 23
  12. 2020EDropout: Energy-Based Dropout and Pruning of Deep Neural Networkscited by 20
  13. 2019Energy-Based Continuous Inverse Optimal Controlcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2025Generative or Discriminative? Revisiting Text Classification in the Era of Transformerscited by 9
  2. 2024CALICO: Confident Active Learning with Integrated Calibrationcited by 1
  3. 2024Pseudo-label Learning with Calibrated Confidence Using an Energy-based Modelcited by 0
  4. 2026Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoringcited by 0