mlsanityData to 14 Oct 2026

PIPer: On-Device Environment Setup via Online Reinforcement Learning

arXiv 2509.25455 ↗First posted 29 Sep 2025cs.LG cs.AI cs.SEcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Environment setup-the process of configuring the system to work with a specific software project-represents a persistent challenge in Software Engineering (SE). Automated environment setup methods could assist developers by providing fully configured environments for arbitrary repositories without manual effort. This also helps SE researchers to scale execution-based benchmarks. However, recent studies reveal that even state-of-the-art Large Language Models (LLMs) achieve limited success in automating this task. To address this limitation, we tune a specialized model for environment setup. We combine supervised fine-tuning for generating correct Bash scripts and Reinforcement Learning with Verifiable Rewards (RLVR) to adapt it to the task of environment setup. On EnvBench-Python, our method enables Qwen3-8B (a model runnable on consumer hardware) to perform on par with larger models-Qwen3-32B and GPT-4o. The training code and model checkpoints are available online: https://github.com/JetBrains-Research/PIPer.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 44 of its 54 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

44 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025RL's Razor: Why Online Reinforcement Learning Forgets Less4 Sep
  2. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification7 Aug
  3. 2025Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning5 Aug
  4. 2025Tool-integrated Reinforcement Learning for Repo Deep Search5 Aug
  5. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training28 Jan

Other references · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  11. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  12. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
27 more
  1. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  2. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  3. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  4. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  5. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  6. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 264
  7. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  8. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
  9. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
  10. 2025SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agentscited by 142
  11. 2025LocAgent: Graph-Guided LLM Agents for Code Localizationcited by 86
  12. 2025SWE-bench Goes Live!cited by 80
  13. 2025Reinforcing General Reasoning without Verifierscited by 77
  14. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 77
  15. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 47
  16. 2024Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Studycited by 44
  17. 2025Repo2Run: Automated Building Executable Environment for Code Repository at Scalecited by 39
  18. 2025EnvBench: A Benchmark for Automated Environment Setupcited by 36
  19. 2025Automated Benchmark Generation for Repository-Level Coding Taskscited by 29
  20. 2025Multi-Turn Code Generation Through Single-Step Rewardscited by 28
  21. 2025Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMscited by 26
  22. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  23. 2024Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projectscited by 21
  24. 2025SweRank: Software Issue Localization with Code Rankingcited by 20
  25. 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environmentscited by 18
  26. 2025Bridging Bug Localization and Issue Fixing: A Hierarchical Localization Framework Leveraging Large Language Modelscited by 11
  27. 2025Assessing Small Language Models for Code Generation: An Empirical Study with Benchmarkscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SETA: Scaling Environments for Terminal Agents
  2. 2026From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
  3. 2026AutoEnvScaling: Automating the Data Flywheel with Terminal Agents
  4. 2026SWE-MiniSandbox: Container-Free Reinforcement Learning for Building Software Engineering Agents
  5. 2025Reinforcement Learning for Machine Learning Engineering Agents
  6. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning
  7. 2026Envs-FORGE: Frontier-Optimized Reward-Grounded Environment Synthesis for Agent RL
  8. 2026ECHO: Terminal Agents Learn World Models for Free