mlsanityData to 7 Oct 2026

Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach

arXiv 2608.15224 ↗First posted 15 Aug 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Reliable post-hoc evaluation asks whether already generated text satisfies a target criterion after generation. In this paper we study a focused frozen-embedding setting using principle-evaluation proxy tasks: toxicity detection, fine-grained emotion categorization, and ordinal review rating. General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples in overlapping regions of the representation space. We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings. The module combines a semantic stream, a prototype-anchor attention stream, supervised contrastive learning, offset-based prototype-margin regularization, and stream regularization to produce a compact task-adapted representation without updating the base encoder. Controlled experiments show that PGCL improves over raw frozen embeddings on all three datasets and gives the clearest direct-baseline margin on AmazonReviews, while remaining competitive with strong direct frozen metric-learning baselines on GoEmotions and ToxicComment. We also add supervised residual-adapter, encoder-LoRA, full fine-tuning, objective ablation, sensitivity, and fully logged few-shot LLM protocol diagnostics to define the boundary of the claim. The theoretical analysis is revised as a sufficient-condition account for prototype-margin behavior under explicit assumptions in the prototype-mapping space, rather than as an unconditional training or final-embedding separation guarantee.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

40 of 54 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2025Qwen3 Technical Reportcited by 8,195
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
28 more
  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  2. 2020Supervised Contrastive Learningcited by 2,803
  3. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  6. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  7. 2021Ethical and social risks of harm from Language Modelscited by 957
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  9. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  10. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  11. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  12. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  13. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  14. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  15. 2023An Overview of Catastrophic AI Riskscited by 214
  16. 2020Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddingscited by 169
  17. 2021Challenges in Detoxifying Language Modelscited by 169
  18. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  19. 2024Towards Robust Graph Incremental Learning on Evolving Graphscited by 18
  20. 2024PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networkscited by 10
  21. 2023On the Limitation and Experience Replay for GNNs in Continual Learningcited by 7
  22. 2019Learning Task-specific Representation for Novel Words in Sequence Labelingcited by 5
  23. 2024Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networkscited by 5
  24. 2024On the Topology Awareness and Generalization Performance of Graph Neural Networkscited by 3
  25. 2025A Non-Asymptotic Convergent Analysis for Scored-Based Graph Generative Model via a System of Stochastic Differential Equationscited by 3
  26. 2024BG-HGNN: Toward Efficient Learning for Complex Heterogeneous Graphscited by 2
  27. 2024Improving Implicit Regularization of SGD with Preconditioning for Least Square Problemscited by 2
  28. 2026Full-Graph vs. Mini-Batch Training: Comprehensive Analysis from a Batch Size and Fan-Out Size Perspectivecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.