# Bayesian Hierarchical Words Representation Learning

arXiv: https://arxiv.org/abs/2004.07126 · HTML: https://mlsanity.com/paper/2004.07126/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2020.

## Built on by (most cited first, 10 of 10)

- [Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps](https://mlsanity.com/paper/2204.11073/) (2022)
- [Deep Integrated Explanations](https://mlsanity.com/paper/2310.15368/) (2023)
- [Interpreting BERT-based Text Similarity via Activation and Saliency Maps](https://mlsanity.com/paper/2208.06612/) (2022)
- [Cold Item Integration in Deep Hybrid Recommenders via Tunable Stochastic Gates](https://mlsanity.com/paper/2112.07615/) (2021)
- [Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models](https://mlsanity.com/paper/2310.16584/) (2023)
- [MetricBERT: Text Representation Learning via Self-Supervised Triplet Training](https://mlsanity.com/paper/2208.06610/) (2022)
- [Representation Learning via Variational Bayesian Networks](https://mlsanity.com/paper/2306.16326/) (2023)
- [Efficient Discovery and Effective Evaluation of Visual Perceptual Similarity: A Benchmark and Beyond](https://mlsanity.com/paper/2308.14753/) (2023)
- [Denoising Word Embeddings by Averaging in a Shared Space](https://mlsanity.com/paper/2106.02954/) (2021)
- [FastDoc: Domain-Specific Fast Continual Pre-training Technique using Document-Level Metadata and Taxonomy](https://mlsanity.com/paper/2306.06190/) (2023)

## Abstract (arXiv)

This paper presents the Bayesian Hierarchical Words Representation (BHWR) learning algorithm. BHWR facilitates Variational Bayes word representation learning combined with semantic taxonomy modeling via hierarchical priors. By propagating relevant information between related words, BHWR utilizes the taxonomy to improve the quality of such representations. Evaluation of several linguistic datasets demonstrates the advantages of BHWR over suitable alternatives that facilitate Bayesian modeling with or without semantic priors. Finally, we further show that BHWR produces better representations for rare words.