# A General Framework for Producing Interpretable Semantic Text Embeddings

arXiv: https://arxiv.org/abs/2410.03435 · HTML: https://mlsanity.com/paper/2410.03435/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 58 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 3 of 3)

- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [BM25S: Orders of magnitude faster lexical search via eager sparse scoring](https://mlsanity.com/paper/2407.03618/) (2024)
- [Crafting Interpretable Embeddings by Asking LLMs Questions](https://mlsanity.com/paper/2405.16714/) (2024)

## Built on by (most cited first, 10 of 10)

- [Bayesian Concept Bottleneck Models with LLM Priors](https://mlsanity.com/paper/2410.15555/) (2024)
- [Interpretable Text Embeddings and Text Similarity Explanation: A Survey](https://mlsanity.com/paper/2502.14862/) (2025)
- [Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning](https://mlsanity.com/paper/2506.08354/) (2025)
- [Human-AI Co-design for Clinical Prediction Models](https://mlsanity.com/paper/2601.09072/) (2026)
- [PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encoder](https://mlsanity.com/paper/2505.24646/) (2025)
- [Very Efficient Listwise Multimodal Reranking for Long Documents](https://mlsanity.com/paper/2605.11864/) (2026)
- [LDIR: Low-Dimensional Dense and Interpretable Text Embeddings with Relative Representations](https://mlsanity.com/paper/2505.10354/) (2025)
- [Reducing Hallucinations in LLM-based Scientific Literature Analysis Using Peer Context Outlier Detection](https://mlsanity.com/paper/2604.01461/) (2026)
- [Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Text](https://mlsanity.com/paper/2603.01690/) (2026)
- [Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Bias](https://mlsanity.com/paper/2607.01973/) (2026)

## Abstract (arXiv)

Semantic text embedding is essential to many tasks in Natural Language Processing (NLP). While black-box models are capable of generating high-quality embeddings, their lack of interpretability limits their use in tasks that demand transparency. Recent approaches have improved interpretability by leveraging domain-expert-crafted or LLM-generated questions, but these methods rely heavily on expert input or well-prompt design, which restricts their generalizability and ability to generate discriminative questions across a wide range of tasks. To address these challenges, we introduce algoCQG-MBQA (Contrastive Question Generation - Multi-task Binary Question Answering), a general framework for producing interpretable semantic text embeddings across diverse tasks. Our framework systematically generates highly discriminative, low cognitive load yes/no questions through the algoCQG method and answers them efficiently with the algoMBQA model, resulting in interpretable embeddings in a cost-effective manner. We validate the effectiveness and interpretability of algoCQG-MBQA through extensive experiments and ablation studies, demonstrating that it delivers embedding quality comparable to many advanced black-box models while maintaining inherently interpretability. Additionally, algoCQG-MBQA outperforms other interpretable text embedding methods across various downstream tasks.