mlsanityData to 14 Oct 2026

Soft Prompts for Evaluation: Measuring Conditional Distance of Capabilities

arXiv 2505.14943 ↗First posted 20 May 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

To help evaluate and understand the latent capabilities of language models, this paper introduces an approach using optimized input embeddings, or 'soft prompts,' as a metric of conditional distance between a model and a target behavior. The technique aims to facilitate latent capability discovery as a part of automated red teaming/evaluation suites and to provide quantitative feedback about the accessibility of potentially concerning behaviors in a way that may scale to powerful future models, including those which may otherwise be capable of deceptive alignment. An evaluation framework using soft prompts is demonstrated in natural language, chess, and pathfinding, and the technique is extended with generalized conditional soft prompts to aid in constructing task evaluations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 23 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

13 of 23 references.

References · 13

  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  8. 2022In-context Learning and Induction Headscited by 964
  9. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  10. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  11. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  12. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 96
1 more
  1. 2023Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Promptingcited by 33

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Structured Prompts Improve Evaluation of Language Models
  2. 2026RepBench: Compiling Benchmarks into Capability Representations for Large Language Models
  3. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
  4. 2025The Elicitation Game: Evaluating Capability Elicitation Techniques
  5. 2026Evaluating 5W3H Structured Prompting for Intent Alignment in Human-AI Interaction
  6. 2025What Makes a Good Natural Language Prompt?
  7. 2025Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages
  8. 2023Establishing Trustworthiness: Rethinking Tasks and Model Evaluation