Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach
Not on the map. Why?
Abstract (arXiv)
Reliable post-hoc evaluation asks whether already generated text satisfies a target criterion after generation. In this paper we study a focused frozen-embedding setting using principle-evaluation proxy tasks: toxicity detection, fine-grained emotion categorization, and ordinal review rating. General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples in overlapping regions of the representation space. We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings. The module combines a semantic stream, a prototype-anchor attention stream, supervised contrastive learning, offset-based prototype-margin regularization, and stream regularization to produce a compact task-adapted representation without updating the base encoder. Controlled experiments show that PGCL improves over raw frozen embeddings on all three datasets and gives the clearest direct-baseline margin on AmazonReviews, while remaining competitive with strong direct frozen metric-learning baselines on GoEmotions and ToxicComment. We also add supervised residual-adapter, encoder-LoRA, full fine-tuning, objective ablation, sensitivity, and fully logged few-shot LLM protocol diagnostics to define the boundary of the claim. The theoretical analysis is revised as a sufficient-condition account for prototype-margin behavior under explicit assumptions in the prototype-mapping space, rather than as an unconditional training or final-embedding separation guarantee.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 15 Aug 2026.
Builds on
40 of 54 references.
References · 40
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2025Qwen3 Technical Reportcited by 8,195
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
28 more
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
- 2020Supervised Contrastive Learningcited by 2,803
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
- 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
- 2022MTEB: Massive Text Embedding Benchmarkcited by 623
- 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
- 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
- 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
- 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
- 2023An Overview of Catastrophic AI Riskscited by 214
- 2020Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddingscited by 169
- 2021Challenges in Detoxifying Language Modelscited by 169
- 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
- 2024Towards Robust Graph Incremental Learning on Evolving Graphscited by 18
- 2024PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networkscited by 10
- 2023On the Limitation and Experience Replay for GNNs in Continual Learningcited by 7
- 2019Learning Task-specific Representation for Novel Words in Sequence Labelingcited by 5
- 2024Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networkscited by 5
- 2024On the Topology Awareness and Generalization Performance of Graph Neural Networkscited by 3
- 2025A Non-Asymptotic Convergent Analysis for Scored-Based Graph Generative Model via a System of Stochastic Differential Equationscited by 3
- 2024BG-HGNN: Toward Efficient Learning for Complex Heterogeneous Graphscited by 2
- 2024Improving Implicit Regularization of SGD with Preconditioning for Least Square Problemscited by 2
- 2026Full-Graph vs. Mini-Batch Training: Comprehensive Analysis from a Batch Size and Fan-Out Size Perspectivecited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.