mlsanityData to 14 Oct 2026

Learning after Execution with Action-Specific Credit

arXiv 2610.14573 ↗First posted 10 Oct 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit-assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image-to-TikZ serves as an executable visual testbed for this setting. On 945 held-out examples, learning after execution strengthens the shared policy under single-call inference and also produces a post-execution corrective capability. Single-call render success reaches 92.70%, compared with 88.25% for continued single-stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single-stage training and depends on the intermediate state. Alternative credit definitions also induce different group-relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available at https://github.com/tianyufengcn/Learning-after-Execution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

26 of 34 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023Teaching Large Language Models to Self-Debugcited by 988
  8. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  9. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  10. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  11. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  12. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
14 more
  1. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  2. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
  3. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
  4. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  5. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
  6. 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 79
  7. 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 75
  8. 2024Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedbackcited by 46
  9. 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 24
  10. 2024Image2Struct: Benchmarking Structure Extraction for Vision-Language Modelscited by 19
  11. 2025Empowering Multi-Turn Tool-Integrated Agentic Reasoning with Group Turn Policy Optimizationcited by 13
  12. 2026MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matchingcited by 10
  13. 2026TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learningcited by 7
  14. 2026Omni-I2C: A Holistic Benchmark for High-Fidelity Image-to-Code Generationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
  2. 2026PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning
  3. 2026Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
  4. 2026Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning
  5. 2026RELACE: retrospective likelihood-based action credit estimation for long-horizon language agents
  6. 2026PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulation
  7. 2026TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning
  8. 2026Trust the Critic More