A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning

Much progress has been made in semi-supervised learning (SSL) by combining methods that exploit different aspects of the data distribution, e.g. consistency regularisation relies on properties of p(x), whereas entropy minimisation pertains to the label distribution p(y|x). Focusing on the latter, we present a probabilistic model for discriminative SSL, that mirrors its classical generative counterpart. Under the assumption y|x is deterministic, the prior over latent variables becomes discrete. We show that several well-known SSL methods can be interpreted as approximating this prior, and can be improved upon. We extend the discriminative model to neuro-symbolic SSL, where label features satisfy logical rules, by showing such rules relate directly to the above prior, thus justifying a family of methods that link statistical learning and logical reasoning, and unifying them with regular SSL.