mlsanityData to 14 Oct 2026

State2State: Environment-Derived Mid-Training for LLM Agents

arXiv 2608.04934 ↗First posted 5 Aug 2026cs.CL cs.LGcited by 2

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Training LLM agents commonly relies on supervised fine-tuning from expert trajectories or online reinforcement learning over human-specified tasks with handcrafted verifiers. Though effective, both remain bottlenecked by externally specified tasks and supervision signals, limiting the scalability and diversity of agent training. We study an environment learning paradigm in which agents acquire interaction and manipulation capabilities solely through environment interaction, without externally specified tasks. We propose State2State, an environment-derived mid-training method that converts explored environment states into training objectives, challenging agents to reach a specified target state. By deriving tasks from environment exploration and verifying success through rule-based state matching, State2State provides scalable and verifiable training objectives without expert supervision or manual task design. Experiments on ALFWorld and ScienceWorld show that State2State improves agent performance as a standalone environment-learning stage in most settings. As initialization for downstream RL, it further improves final performance and learning efficiency, with promising evidence of cross-environment generalization.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 45 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

29 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective16 Oct
  2. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May
  3. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr

Other references · 26

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  6. 2017Hindsight Experience Replaycited by 1,136
  7. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  8. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  9. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  10. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  11. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  12. 2025AgentEvolver: Towards Efficient Self-Evolving Agent Systemcited by 85
14 more
  1. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 83
  2. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 82
  3. 2025MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environmentscited by 72
  4. 2025RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agentscited by 58
  5. 2025TaskCraft: Automated Generation of Agentic Taskscited by 51
  6. 2025Scaling Agents via Continual Pre-trainingcited by 40
  7. 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGIcited by 36
  8. 2026Reinforcement World Model Learning for LLM-based Agentscited by 24
  9. 2025A Survey on LLM Mid-Trainingcited by 21
  10. 2026daVinci-Dev: Agent-native Mid-training for Software Engineeringcited by 18
  11. 2025Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Maskingcited by 17
  12. 2025SAND: Boosting LLM Agents with Self-Taught Action Deliberationcited by 14
  13. 2025GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoningcited by 13
  14. 2026AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabelingcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 3
  2. 2026Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EAU: Environment-Agent-User Co-evolution for Training User-Centric Agents
  2. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents
  3. 2026Can RL Improve Generalization of LLM Agents? An Empirical Study
  4. 2025Don't Just Fine-tune the Agent, Tune the Environment
  5. 2026From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
  6. 2026EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
  7. 2025Simulating Environments with Reasoning Models for Agent Training
  8. 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement