# Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data

arXiv: https://arxiv.org/abs/2506.23182 · HTML: https://mlsanity.com/paper/2506.23182/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 94 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [xLSTM: Extended Long Short-Term Memory](https://mlsanity.com/paper/2405.04517/) (2024)
- [Learning from positive and unlabeled data: a survey](https://mlsanity.com/paper/1811.04820/) (2018)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [Protein structure generation via folding diffusion](https://mlsanity.com/paper/2209.15611/) (2022)
- [Towards Visually Explaining Variational Autoencoders](https://mlsanity.com/paper/1911.07389/) (2019)
- [Conditional Antibody Design as 3D Equivariant Graph Translation](https://mlsanity.com/paper/2208.06073/) (2022)
- [Modern Hopfield Networks and Attention for Immune Repertoire Classification](https://mlsanity.com/paper/2007.13505/) (2020)
- [Sequential Integrated Gradients: a simple but effective method for explaining language models](https://mlsanity.com/paper/2305.15853/) (2023)
- [Protein sequence design with deep generative models](https://mlsanity.com/paper/2104.04457/) (2021)
- [Generating and designing DNA with deep generative models](https://mlsanity.com/paper/1712.06148/) (2017)
- [A Survey of Generative AI for de novo Drug Design: New Frontiers in Molecule and Protein Generation](https://mlsanity.com/paper/2402.08703/) (2024)
- [Explaining and Interpreting LSTMs](https://mlsanity.com/paper/1909.12114/) (2019)
- [Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda](https://mlsanity.com/paper/2404.09554/) (2024)
- [Understanding Integrated Gradients with SmoothTaylor for Deep Neural Network Attribution](https://mlsanity.com/paper/2004.10484/) (2020)
- [Evaluating the Interpretability of Generative Models by Interactive Reconstruction](https://mlsanity.com/paper/2102.01264/) (2021)
- [Interpretable Deep Learning in Drug Discovery](https://mlsanity.com/paper/1903.02788/) (2019)
- [ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models](https://mlsanity.com/paper/2402.00794/) (2024)
- [Explainable AI for Bioinformatics: Methods, Tools, and Applications](https://mlsanity.com/paper/2212.13261/) (2022)
- [Multi-Level Explanations for Generative Language Models](https://mlsanity.com/paper/2403.14459/) (2024)
- [Unsupervised Representation Learning of DNA Sequences](https://mlsanity.com/paper/1906.03087/) (2019)
- [ImmunoLingo: Linguistics-based formalization of the antibody language](https://mlsanity.com/paper/2209.12635/) (2022)
- [Explaining Pre-Trained Language Models with Attribution Scores: An Analysis in Low-Resource Settings](https://mlsanity.com/paper/2403.05338/) (2024)

## Abstract (arXiv)

Generative machine learning models offer a powerful framework for therapeutic design by efficiently exploring large spaces of biological sequences enriched for desirable properties. Unlike supervised learning methods, which require both positive and negative labeled data, generative models such as LSTMs can be trained solely on positively labeled sequences, for example, high-affinity antibodies. This is particularly advantageous in biological settings where negative data are scarce, unreliable, or biologically ill-defined. However, the lack of attribution methods for generative models has hindered the ability to extract interpretable biological insights from such models. To address this gap, we developed Generative Attribution Metric Analysis (GAMA), an attribution method for autoregressive generative models based on Integrated Gradients. We assessed GAMA using synthetic datasets with known ground truths to characterize its statistical behavior and validate its ability to recover biologically relevant features. We further demonstrated the utility of GAMA by applying it to experimental antibody-antigen binding data. GAMA enables model interpretability and the validation of generative sequence design strategies without the need for negative training data.