# Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations

arXiv: https://arxiv.org/abs/2311.04335 · HTML: https://mlsanity.com/paper/2311.04335/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 30 of 36)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [Sparse, Dense, and Attentional Representations for Text Retrieval](https://mlsanity.com/paper/2005.00181/) (2020)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [WiCE: Real-World Entailment for Claims in Wikipedia](https://mlsanity.com/paper/2303.01432/) (2023)
- [Complex Claim Verification with Evidence Retrieved in the Wild](https://mlsanity.com/paper/2305.11859/) (2023)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims](https://mlsanity.com/paper/1906.03538/) (2019)
- [Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index](https://mlsanity.com/paper/1906.05807/) (2019)
- [Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters](https://mlsanity.com/paper/2204.07447/) (2022)
- [Multi-View Document Representation Learning for Open-Domain Dense Retrieval](https://mlsanity.com/paper/2203.08372/) (2022)
- [Phrase Retrieval Learns Passage Retrieval, Too](https://mlsanity.com/paper/2109.08133/) (2021)
- [PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition](https://mlsanity.com/paper/2212.10750/) (2022)
- [Chain-of-Skills: A Configurable Model for Open-domain Question Answering](https://mlsanity.com/paper/2305.03130/) (2023)

## Built on by (most cited first, 20 of 20)

- [Dense X Retrieval: What Retrieval Granularity Should We Use?](https://mlsanity.com/paper/2312.06648/) (2023)
- [LLatrieval: LLM-Verified Retrieval for Verifiable Generation](https://mlsanity.com/paper/2311.07838/) (2023)
- [A Closer Look at Claim Decomposition](https://mlsanity.com/paper/2403.11903/) (2024)
- [DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation](https://mlsanity.com/paper/2412.13175/) (2024)
- [MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers](https://mlsanity.com/paper/2506.15862/) (2025)
- [Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Use](https://mlsanity.com/paper/2505.02164/) (2025)
- [Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challenges](https://mlsanity.com/paper/2406.06938/) (2024)
- [Sim-GPT: Text Similarity via GPT Annotated Data](https://mlsanity.com/paper/2312.05603/) (2023)
- [A Claim Decomposition Benchmark for Long-form Answer Verification](https://mlsanity.com/paper/2410.12558/) (2024)
- [Educational Cone Model in Embedding Vector Spaces](https://mlsanity.com/paper/2512.04227/) (2025)
- [Evaluate Summarization in Fine-Granularity: Auto Evaluation with LLM](https://mlsanity.com/paper/2412.19906/) (2024)
- [On Reference (In-)Determinacy in Natural Language Inference](https://mlsanity.com/paper/2502.05793/) (2025)
- [InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding](https://mlsanity.com/paper/2509.16534/) (2025)
- [CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis](https://mlsanity.com/paper/2606.19819/) (2026)
- [AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings](https://mlsanity.com/paper/2405.15028/) (2024)
- [Enhancing Answer Attribution for Faithful Text Generation with Large Language Models](https://mlsanity.com/paper/2410.17112/) (2024)
- [Building Open-Retrieval Conversational Question Answering Systems by Generating Synthetic Data and Decontextualizing User Questions](https://mlsanity.com/paper/2507.04884/) (2025)
- [A Navigational Approach for Comprehensive RAG via Traversal over Proposition Graphs](https://mlsanity.com/paper/2601.04859/) (2026)
- [GRASP: Graph Agentic Search over Propositions for Multi-hop Question Answering](https://mlsanity.com/paper/2605.16598/) (2026)
- [Beyond Black Boxes: Visual Grounding for Text Embeddings](https://mlsanity.com/paper/2610.18047/) (2026)

## Abstract (arXiv)

We introduce sub-sentence encoder, a contrastively-learned contextual embedding model for fine-grained semantic representation of text. In contrast to the standard practice with sentence embeddings, where the meaning of an entire sequence of text is encoded into a fixed-length vector, the sub-sentence encoder learns to produce distinct contextual embeddings corresponding to different atomic propositions, i.e. atomic units of meaning expressed within a text sequence. The sub-sentence embeddings are contrastively learned to recognize (inferred) semantic equivalence between propositions across different text sequences. Our experiments show the effectiveness of sub-sentence encoders in applications, such as retrieving supporting facts for fine-grained text attribution or recognizing the conditional semantic similarity between texts. In practice, we demonstrate that sub-sentence encoders keep the same level of inference cost and space complexity compared to sentence encoders.