mlsanityData to 14 Oct 2026

Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text

arXiv 2605.29076 ↗First posted 27 May 2026cs.CL cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

LLMs have advanced text classification, yet existing paradigms face a trade-off: supervised (label only) fine-tuning is scalable but offers limited reasoning on complex text and lacks broader model transparency, while discrete prompt optimization offers human-readable instructions but struggles with performance and scalability. We introduce eXTC (eXplainable Text Classifier) with three progressive stages: (1) learning a Standard Operating Procedure (SOP, or rulebook) in natural language via a new Structured Prompt Optimization algorithm; (2) SOP-grounded reasoning distillation from a large teacher LLM into a compact LM; and (3) expanding reasoning capabilities beyond the initial SOP via reinforcement learning. This design enables eXTC to provide (i) fast inference via a compact LM, with (ii) inference-time local reasoning traces, alongside a global, modular explanation of its learned domain rules, while (iii) significantly outperforming existing paradigms across diverse benchmarks in both classification performance and explanation quality, with stage-by-stage gains.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 34 of its 63 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

34 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning9 Feb

Other references · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  9. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  10. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  11. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  12. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
21 more
  1. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  2. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  3. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  4. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  5. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
  6. 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 148
  7. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 137
  8. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  9. 2023SALMON: Self-Alignment with Instructable Reward Modelscited by 55
  10. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 47
  11. 2025Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussionscited by 23
  12. 2016Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Recordscited by 20
  13. 2024Verbalized Machine Learning: Revisiting Machine Learning with Language Modelscited by 17
  14. 2024Enabling Discriminative Reasoning in LLMs for Legal Judgment Predictioncited by 17
  15. 2017Explaining Anomalies in Groups with Characterizing Subspace Rulescited by 13
  16. 2025Text Classification in the LLM Era -- Where do we stand?cited by 10
  17. 2024Large Language Models are Interpretable Learnerscited by 7
  18. 2025Two-Stage Reasoning-Infused Learning: Improving Classification with LLM-Generated Reasoningcited by 4
  19. 2025Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programscited by 3
  20. 2025Learning to Explain: Prototype-Based Surrogate Models for LLM Classificationcited by 3
  21. 2025RIMRULE: Improving Tool-Using Language Agents via MDL-Guided Rule Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PRL: Prompts from Reinforcement Learning
  2. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
  3. 2026Optimizing LLM Prompt Engineering with DSPy Based Declarative Learning
  4. 2025GTA: Supervised-Guided Reinforcement Learning for Text Classification with Large Language Models
  5. 2023Text Classification via Large Language Models
  6. 2026Automatic Prompt Optimization for Dataset-Level Feature Discovery
  7. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Models
  8. 2025CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs