# RevUp: Revise and Update Information Bottleneck for Event Representation

arXiv: https://arxiv.org/abs/2205.12248 · HTML: https://mlsanity.com/paper/2205.12248/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 24 May 2022.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [A Mutual Information Maximization Perspective of Language Representation Learning](https://mlsanity.com/paper/1910.08350/) (2019)
- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Cross-View Cross-Scene Multi-View Crowd Counting](https://mlsanity.com/paper/2205.01551/) (2022)
- [LOME: Large Ontology Multilingual Extraction](https://mlsanity.com/paper/2101.12175/) (2021)
- [Fine-Grained Zero-Shot Learning with DNA as Side Information](https://mlsanity.com/paper/2109.14133/) (2021)
- [Learning Multimodal VAEs through Mutual Supervision](https://mlsanity.com/paper/2106.12570/) (2021)
- ["What Are You Trying to Do?" Semantic Typing of Event Processes](https://mlsanity.com/paper/2010.06724/) (2020)
- [Event Representations with Tensor-based Compositions](https://mlsanity.com/paper/1711.07611/) (2017)
- [Variational Sequential Labelers for Semi-Supervised Learning](https://mlsanity.com/paper/1906.09535/) (2019)
- [Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering](https://mlsanity.com/paper/2203.07633/) (2022)
- [Hierarchical Quantized Representations for Script Generation](https://mlsanity.com/paper/1808.09542/) (2018)
- [Event Representation with Sequential, Semi-Supervised Discrete Variables](https://mlsanity.com/paper/2010.04361/) (2020)
- [SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations](https://mlsanity.com/paper/2011.10684/) (2020)
- [SURFACE: Semantically Rich Fact Validation with Explanations](https://mlsanity.com/paper/1810.13223/) (2018)
- [Interpretable Deep Representation Learning from Temporal Multi-view Data](https://mlsanity.com/paper/2005.05210/) (2020)
- [Regularized Sequential Latent Variable Models with Adversarial Neural Networks](https://mlsanity.com/paper/2108.04496/) (2021)

## Abstract (arXiv)

The existence of external (``side'') semantic knowledge has been shown to result in more expressive computational event models. To enable the use of side information that may be noisy or missing, we propose a semi-supervised information bottleneck-based discrete latent variable model. We reparameterize the model's discrete variables with auxiliary continuous latent variables and a light-weight hierarchical structure. Our model is learned to minimize the mutual information between the observed data and optional side knowledge that is not already captured by the new, auxiliary variables. We theoretically show that our approach generalizes past approaches, and perform an empirical case study of our approach on event modeling. We corroborate our theoretical results with strong empirical experiments, showing that the proposed method outperforms previous proposed approaches on multiple datasets.