mlsanityData to 7 Oct 2026

Two-Stage Prompt Optimization for Few-Shot Relation Extraction: From Reasoning-Guided Search to Gradient-Guided Refinement

arXiv 2606.29639 ↗First posted 28 Jun 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Automatic prompt optimization is still underexplored for episodic few-shot relation extraction with smaller language models. We propose a two-stage framework that combines reasoning-based prompt optimization with gradient-based prompt optimization. The first stage can use any reasoning-based optimizer to make broadprompt improvements in natural language. The second stage applies our GradPO, which uses loss and gradient signals to identify high-impact prompt spans and refine them with local edits. Experiments on FS-TACRED and FS-FewRel show that local refinement usually improves prompts found by the first stage, and GradPO is the most consistent refiner. Our framework achieves state-of-the-art performance on FS-TACRED with Qwen3-4B and remains competitive on FS-FewRel.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 28 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

17 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers12 Dec
  2. 2023Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction10 Nov

Other references · 15

  1. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  3. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  4. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  5. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  6. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  7. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  8. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  9. 2023How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?cited by 30
  10. 2021Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemescited by 27
  11. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 22
  12. 2024Best of Both Worlds: A Pliable and Generalizable Neuro-Symbolic Approach for Relation Classificationcited by 6
3 more
  1. 2025Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Studycited by 6
  2. 2025Local Prompt Optimizationcited by 4
  3. 2026Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extractioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Revisiting Large Language Models as Zero-shot Relation Extractors
  2. 2024Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
  3. 2026How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs
  4. 2023Revisiting Relation Extraction in the era of Large Language Models
  5. 2023Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors
  6. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
  7. 2022RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction
  8. 2025TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards