mlsanityData to 14 Oct 2026

TURBO: The Swiss Knife of Auto-Encoders

arXiv 2311.06527 ↗First posted 11 Nov 2023cs.LG cs.CR cs.ITcited by 3

Not on the map. Why?

Abstract (arXiv)

We present a novel information-theoretic framework, termed as TURBO, designed to systematically analyse and generalise auto-encoding methods. We start by examining the principles of information bottleneck and bottleneck-based networks in the auto-encoding setting and identifying their inherent limitations, which become more prominent for data with multiple relevant, physics-related representations. The TURBO framework is then introduced, providing a comprehensive derivation of its core concept consisting of the maximisation of mutual information between various data representations expressed in two directions reflecting the information flows. We illustrate that numerous prevalent neural network models are encompassed within this framework. The paper underscores the insufficiency of the information bottleneck concept in elucidating all such models, thereby establishing TURBO as a preferable theoretical reference. The introduction of TURBO contributes to a richer understanding of data representation and the structure of neural network models, enabling more efficient and versatile applications.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 11 Nov 2023.

Builds on

26 of 43 references.

References · 26

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  5. 2015Variational Inference with Normalizing Flowscited by 2,170
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  8. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  9. 2016Deep Variational Information Bottleneckcited by 930
  10. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  11. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  12. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
14 more
  1. 2015Adversarial Autoencoderscited by 364
  2. 2020Adversarial Latent Autoencoderscited by 155
  3. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 132
  4. 2016Learning in Implicit Generative Modelscited by 125
  5. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  6. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  7. 2018Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principlecited by 70
  8. 2019Information bottleneck through variational glassescited by 19
  9. 2022Unsupervised Anomaly Detection in Medical Images with a Memory-augmented Multi-level Cross-attentional Masked Autoencodercited by 15
  10. 2022Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utilitycited by 12
  11. 2019Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embeddingcited by 8
  12. 2023Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detectioncited by 7
  13. 2021Turbo-Sim: a generalised generative model with a physical latent spacecited by 5
  14. 2022Digital twins of physical printing-imaging channelcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024PIPPIN: Generating variable length full events from partonscited by 6
  2. 2025TRANSIT your events into a new mass: Fast background interpolation for weakly-supervised anomaly searchescited by 0
  3. 2025TwinTURBO: Semi-Supervised Fine-Tuning of Foundation Models via Mutual Information Decompositions for Downstream Task and Latent Spacescited by 0