# Hyper-CL: Conditioning Sentence Representations with Hypernetworks

arXiv: https://arxiv.org/abs/2403.09490 · HTML: https://mlsanity.com/paper/2403.09490/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer](https://mlsanity.com/paper/2202.02113/) (2022)

## Built on by (most cited first, 8 of 8)

- [Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning](https://mlsanity.com/paper/2506.08354/) (2025)
- [Out-of-the-Box Conditional Text Embeddings from Large Language Models](https://mlsanity.com/paper/2504.16411/) (2025)
- [Individualised Treatment Effects Estimation with Composite Treatments and Composite Outcomes](https://mlsanity.com/paper/2502.08282/) (2025)
- [CASE -- Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement](https://mlsanity.com/paper/2503.17279/) (2025)
- [Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Models](https://mlsanity.com/paper/2509.14399/) (2025)
- [PoLi-RL: A Point-to-List Reinforcement Learning Framework for Conditional Semantic Textual Similarity](https://mlsanity.com/paper/2510.04080/) (2025)
- [Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMs](https://mlsanity.com/paper/2609.32684/) (2026)
- [An open dataset of neural networks for hypernetwork research](https://mlsanity.com/paper/2507.15869/) (2025)

## Abstract (arXiv)

While the introduction of contrastive learning frameworks in sentence representation learning has significantly contributed to advancements in the field, it still remains unclear whether state-of-the-art sentence embeddings can capture the fine-grained semantics of sentences, particularly when conditioned on specific perspectives. In this paper, we introduce Hyper-CL, an efficient methodology that integrates hypernetworks with contrastive learning to compute conditioned sentence representations. In our proposed approach, the hypernetwork is responsible for transforming pre-computed condition embeddings into corresponding projection layers. This enables the same sentence embeddings to be projected differently according to various conditions. Evaluation on two representative conditioning benchmarks, namely conditional semantic text similarity and knowledge graph completion, demonstrates that Hyper-CL is effective in flexibly conditioning sentence representations, showcasing its computational efficiency at the same time. We also provide a comprehensive analysis of the inner workings of our approach, leading to a better interpretation of its mechanisms.