# Abundance-Aware Set Transformer for Microbiome Sample Embedding

arXiv: https://arxiv.org/abs/2508.11075 · HTML: https://mlsanity.com/paper/2508.11075/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 38 references that are in the map.
- First posted on arXiv 14 Aug 2025.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)

## Built on by (most cited first, 1 of 1)

- [Set-Aggregated Genome Embeddings for Microbiome Abundance Prediction](https://mlsanity.com/paper/2605.12286/) (2026)

## Abstract (arXiv)

Microbiome sample representation to input into LLMs is essential for downstream tasks such as phenotype prediction and environmental classification. While prior studies have explored embedding-based representations of each microbiome sample, most rely on simple averaging over sequence embeddings, often overlooking the biological importance of taxa abundance. In this work, we propose an abundance-aware variant of the Set Transformer to construct fixed-size sample-level embeddings by weighting sequence embeddings according to their relative abundance. Without modifying the model architecture, we replicate embedding vectors proportional to their abundance and apply self-attention-based aggregation. Our method outperforms average pooling and unweighted Set Transformers on real-world microbiome classification tasks, achieving perfect performance in some cases. These results demonstrate the utility of abundance-aware aggregation for robust and biologically informed microbiome representation. To the best of our knowledge, this is one of the first approaches to integrate sequence-level abundance into Transformer-based sample embeddings.