mlsanityData to 7 Oct 2026

SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation

arXiv 2603.16161 ↗First posted 17 Mar 2026cs.AIcited by 3

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Agentic Reinforcement Learning (RL) shows promise for complex tasks, but Text-to-SQL remains mostly restricted to single-turn paradigms. A primary bottleneck is the credit assignment problem. In traditional paradigms, rewards are determined solely by the final-turn feedback, which ignores the intermediate process and leads to ambiguous credit evaluation. To address this, we propose Agentic SQL, a framework featuring a universal two-tiered reward mechanism designed to provide effective trajectory-level evaluation and dense step-level signals. First, we introduce Aggregated Trajectory Reward (ATR) to resolve multi-turn credit assignment. Using an asymmetric transition matrix, ATR aggregates process-oriented scores to incentivize continuous improvement. Leveraging Lyapunov stability theory, we prove ATR acts as an energy dissipation operator, guaranteeing a cycle-free policy and monotonic convergence. Second, Column-Set Matching Reward (CSMR) provides immediate step-level rewards to mitigate sparsity. By executing queries at each turn, CSMR converts binary (0/1) feedback into dense [0, 1] signals based on partial correctness. Evaluations on BIRD show a 5% gain over binary-reward GRPO. Notably, our approach outperforms SOTA Arctic-Text2SQL-R1-7B on BIRD and Spider 2.0 using identical models, propelling Text-to-SQL toward a robust multi-turn agent paradigm.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 52 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

37 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MTIR-SQL: Multi-turn Tool-Integrated Reasoning Reinforcement Learning for Text-to-SQL29 Oct
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey2 Sep
  3. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr

Other references · 34

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  3. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  4. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  5. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  6. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  7. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  8. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  9. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  10. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  11. 2025SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learningcited by 67
  12. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
22 more
  1. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 40
  2. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
  3. 2025MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosiscited by 32
  4. 2026Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RLcited by 27
  5. 2025Bottom-up Policy Optimization: Your Language Model Policy Secretly Contains Internal Policiescited by 22
  6. 2026EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selectioncited by 22
  7. 2025Reasoning-Table: Exploring Reinforcement Learning for Table Reasoningcited by 18
  8. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 18
  9. 2025BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactionscited by 17
  10. 2025STaR-SQL: Self-Taught Reasoner for Text-to-SQLcited by 16
  11. 2024CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editionscited by 13
  12. 2024Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queriescited by 12
  13. 2018Lyapunov Functions for First-Order Methods: Tight Automated Convergence Guaranteescited by 11
  14. 2024How Do Humans Write Code? Large Models Do It the Same Way Toocited by 11
  15. 2024FLEX: Expert-level False-Less EXecution Metric for Reliable Text-to-SQL Benchmarkcited by 11
  16. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedbackcited by 11
  17. 2026VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reductioncited by 9
  18. 2025Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attackscited by 8
  19. 2026PRPO: Aligning Process Reward with Outcome Reward in Policy Optimizationcited by 7
  20. 2025SPFT-SQL: Enhancing Large Language Model for Text-to-SQL Parsing by Self-Play Fine-Tuningcited by 5
  21. 2025MTSQL-R1: Towards Long-Horizon Multi-Turn Text-to-SQL via Agentic Trainingcited by 5
  22. 2025A State-of-the-Art SQL Reasoning Model using RLVRcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026SPPO: Sequence-Level PPO for Long-Horizon Reasoning Taskscited by 7
  2. 2026Recursive Self-Evolving Agents via Held-Out Selectioncited by 6
  3. 2026SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQL
  2. 2026Coverage, Not Targeting: A Structural Regime in Multi-Turn Agent Credit Assignment
  3. 2026TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
  4. 2026AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learning
  5. 2026VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents
  6. 2026Agentic Reinforcement Learning with Self-Distilled Reward Shaping
  7. 2026ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL
  8. 2025Agentic Reinforcement Learning with Implicit Step Rewards