A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
Not on the map. Why?
Abstract (arXiv)
Posterior collapse in variational autoencoders is often diagnosed by its symptoms: a small KL term, a strong decoder, or weak use of the latent code. These signals are useful, but they do not define a collapse boundary. We study a concrete failure mode, input-independent constant collapse, and show that this case admits an exact threshold. For any fixed nonconstant teacher distribution (T(·mid x)), the best constant student is the dataset-average teacher distribution, and its alignment cost is the teacher mutual information (IT(X;T)). Therefore, if a strictly latent-only raw witness achieves alignment loss below this value, with a safety margin, the witness cannot be constant in the input. This identity turns a qualitative failure mode into a measurable one. In CIFAR-100 experiments with per-seed teacher search, full training stays on the certified side of the boundary, removing alignment drives the raw witness into the constant-student regime, and restarting from a collapsed checkpoint with alignment enabled restores the certificate. Tiny-ImageNet-200 fixed-target runs show the same prevention--collapse--rescue pattern across three independently searched teachers. Standard VAE-style baselines, including methods that preserve reconstruction quality or post-hoc predictability, remain negative under the raw certificate. The guarantee is intentionally narrow: it certifies that the matched nonconstant teacher-relative variation passes through the latent pathway, rather than claiming that all forms of posterior collapse have been ruled out.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 5 of its 8 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
5 of 8 references.
References · 5
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2015Generating Sentences from a Continuous Spacecited by 612
- 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 220
- 2016Variational Lossy Autoencodercited by 176
- 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
Built on by
Citing papers per year: 2026: 1.
1 paper.