mlsanityData to 14 Oct 2026

RST: certifying and constructing prescribed information in variational autoencoders

arXiv 2605.18224 ↗First posted 18 May 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Posterior-collapse diagnostics can show that a VAE uses its latent representation, but they do not determine whether a specified information view crosses a representation/readout interface fixed before training. Reconstruction-Student-Teacher (RST) makes this stronger requirement explicit: Reconstruction supplies the VAE objective, the Student is a fixed witness, and the Teacher is a frozen information specification. We derive a teacher-relative certificate showing that a positive margin guarantees transmission of the declared information through the prescribed interface, while every input-independent representation lies at or below the boundary. A centered regular-simplex witness turns this certificate into an explicit construction: a minimum-dimensional closed-form teacher code, its complete affine solution fiber, witness-visible and witness-null routing, a margin-energy path, function-space departure from the centered input-independent point, preservation cylinders, and orthogonal multi-view composition. Experiments on five datasets test boundary crossing, stronger KL pressure, teacher counterfactuals, null routing, escape, preservation, cross-seed reuse, and cross-architecture reuse. On CIFAR-100, auxiliary learned readers remain natively decodable but fail under no-refit transfer, whereas the same fixed RST witness directly reads independently trained ConvVAE and ResVAE encoders. Affine adapters recover transferred auxiliary readers, localizing much of the failure to interface-coordinate mismatch rather than absence of information. These results support prescribed alignment as distinct from ordinary decodability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 43 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

31 of 43 references.

References · 31

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2020Concept Bottleneck Modelscited by 1,032
  5. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  6. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  7. 2019Designing and Interpreting Probes with Control Taskscited by 640
  8. 2015Generating Sentences from a Continuous Spacecited by 612
  9. 2019On Variational Bounds of Mutual Informationcited by 559
  10. 2017Wasserstein Auto-Encoderscited by 373
  11. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  12. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
19 more
  1. 2016Variational Lossy Autoencodercited by 176
  2. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  3. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  4. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  5. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  6. 2020Towards Backward-Compatible Representation Learningcited by 75
  7. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  8. 2019The Usual Suspects? Reassessing Blame for VAE Posterior Collapsecited by 52
  9. 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
  10. 2018Fix your classifier: the marginal value of training the last weight layercited by 43
  11. 2021Regular Polytope Networkscited by 24
  12. 2023Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severitycited by 13
  13. 2023Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealingcited by 12
  14. 2024Stationary Representations: Optimally Approximating Compatibility and Implications for Improved Model Replacementscited by 8
  15. 2025boldsymbolλ-Orthogonality Regularization for Compatible Representation Learningcited by 8
  16. 2023Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Networkcited by 7
  17. 2023Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoderscited by 7
  18. 2025Posterior Collapse as a Phase Transition in Variational Autoencoderscited by 4
  19. 2026A Testable Certificate for Constant Collapse in Teacher-Guided VAEscited by 1

Built on by

Not cited yet.