# Stochastic Bottleneck: Rateless Auto-Encoder for Flexible Dimensionality Reduction

arXiv: https://arxiv.org/abs/2005.02870 · HTML: https://mlsanity.com/paper/2005.02870/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2020.

## Built on by (most cited first, 13 of 13)

- [Learning Adaptive and Temporally Causal Video Tokenization in a 1D Latent Space](https://mlsanity.com/paper/2505.17011/) (2025)
- [ProgDTD: Progressive Learned Image Compression with Double-Tail-Drop Training](https://mlsanity.com/paper/2305.02145/) (2023)
- [Semantic Edge Computing and Semantic Communications in 6G Networks: A Unifying Survey and Research Challenges](https://mlsanity.com/paper/2411.18199/) (2024)
- [One Model, Many Budgets: Elastic Latent Interfaces for Diffusion Transformers](https://mlsanity.com/paper/2603.12245/) (2026)
- [Progressive Neural Compression for Adaptive Image Offloading under Timing Constraints](https://mlsanity.com/paper/2310.05306/) (2023)
- [quEEGNet: Quantum AI for Biosignal Processing](https://mlsanity.com/paper/2210.00864/) (2022)
- [Rateless Joint Source-Channel Coding, and a Blueprint for 6G Semantic Communications System Design](https://mlsanity.com/paper/2502.06095/) (2025)
- [LimitNet: Progressive, Content-Aware Image Offloading for Extremely Weak Devices & Networks](https://mlsanity.com/paper/2504.13736/) (2025)
- [FlexLAM: Resolving the Bottleneck Trade-off in Latent Action Learning](https://mlsanity.com/paper/2606.19408/) (2026)
- [Rateless Autoencoder Codes: Trading off Decoding Delay and Reliability](https://mlsanity.com/paper/2301.12231/) (2023)
- [ProDAT: Progressive Density-Aware Tail-Drop for Point Cloud Coding](https://mlsanity.com/paper/2510.17068/) (2025)
- [Semantic Allocation in Ordered Bottlenecks: Predictive Residual Inference for Visual Representation Learning](https://mlsanity.com/paper/2606.25232/) (2026)
- [Training-Free Bottleneck Width Planning for Convolutional Autoencoders](https://mlsanity.com/paper/2609.30755/) (2026)

## Abstract (arXiv)

We propose a new concept of rateless auto-encoders (RL-AEs) that enable a flexible latent dimensionality, which can be seamlessly adjusted for varying distortion and dimensionality requirements. In the proposed RL-AEs, instead of a deterministic bottleneck architecture, we use an over-complete representation that is stochastically regularized with weighted dropouts, in a manner analogous to sparse AE (SAE). Unlike SAEs, our RL-AEs employ monotonically increasing dropout rates across the latent representation nodes such that the latent variables become sorted by importance like in principal component analysis (PCA). This is motivated by the rateless property of conventional PCA, where the least important principal components can be discarded to realize variable rate dimensionality reduction that gracefully degrades the distortion. In contrast, since the latent variables of conventional AEs are equally important for data reconstruction, they cannot be simply discarded to further reduce the dimensionality after the AE model is trained. Our proposed stochastic bottleneck framework enables seamless rate adaptation with high reconstruction performance, without requiring predetermined latent dimensionality at training. We experimentally demonstrate that the proposed RL-AEs can achieve variable dimensionality reduction while achieving low distortion compared to conventional AEs.