# Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Text

arXiv: https://arxiv.org/abs/2603.01690 · HTML: https://mlsanity.com/paper/2603.01690/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [A Question-Entailment Approach to Question Answering](https://mlsanity.com/paper/1901.08079/) (2019)
- [BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers](https://mlsanity.com/paper/2404.18443/) (2024)
- [R2MED: A Benchmark for Reasoning-Driven Medical Retrieval](https://mlsanity.com/paper/2505.14558/) (2025)
- [Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature](https://mlsanity.com/paper/2408.15836/) (2024)
- [Interpretable Text Embeddings and Text Similarity Explanation: A Survey](https://mlsanity.com/paper/2502.14862/) (2025)
- [A General Framework for Producing Interpretable Semantic Text Embeddings](https://mlsanity.com/paper/2410.03435/) (2024)
- [HunFlair2 in a cross-corpus evaluation of biomedical named entity recognition and normalization tools](https://mlsanity.com/paper/2402.12372/) (2024)
- [Biomedical Interpretable Entity Representations](https://mlsanity.com/paper/2106.09502/) (2021)

## Abstract (arXiv)

While dense biomedical embeddings achieve strong performance, their opaque dimensions limit transparency in biomedical NLP. Recent question-based interpretable embeddings represent text through binary answers to natural-language questions, but existing approaches rely primarily on corpus-driven signals, often capturing topical or stylistic differences rather than fine-grained biomedical distinctions. We propose QIME, an ontology-grounded framework for interpretable biomedical text embeddings in which each dimension corresponds to an explicit biomedical-domain yes/no question. QIME leverages a biomedical ontology to guide contrastive question generation from semantic clusters, producing atomic, domain-grounded questions. It constructs embeddings via similarity-based semantic activation with MMR-based diversity-aware dimension selection, yielding sparse representations efficiently. Experiments on biomedical clustering, STS and retrieval benchmarks show that QIME consistently outperforms prior interpretable embedding methods and substantially narrows the gap to strong black-box biomedical encoders. It also imposes substantially lower cognitive burden than existing methods, providing concise and domain-specific interpretations. The code is available at https://github.com/L1nzh/QIME.