mlsanityData to 14 Oct 2026

Prolonging Tool Life: Learning Skillful Use of General-purpose Tools through Lifespan-guided Reinforcement Learning

arXiv 2507.17275 ↗First posted 23 Jul 2025cs.RO cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

In inaccessible environments with uncertain task demands, robots often rely on general-purpose tools that lack predefined usage strategies. These tools are not tailored for particular operations, making their longevity highly sensitive to how they are used. This creates a fundamental challenge: how can a robot learn a tool-use policy that both completes the task and prolongs the tool's lifespan? In this work, we address this challenge by introducing a reinforcement learning (RL) framework that incorporates tool lifespan as a factor during policy optimization. Our framework leverages Finite Element Analysis (FEA) and Miner's Rule to estimate Remaining Useful Life (RUL) based on accumulated stress, and integrates the RUL into the RL reward to guide policy learning toward lifespan-guided behavior. To handle the fact that RUL can only be estimated after task execution, we introduce an Adaptive Reward Normalization (ARN) mechanism that dynamically adjusts reward scaling based on estimated RULs, ensuring stable learning signals. We validate our method across simulated and real-world tool use tasks, including Object-Moving and Door-Opening with multiple general-purpose tools. The learned policies consistently prolong tool lifespan (up to 8.01x in simulation) and transfer effectively to real-world settings, demonstrating the practical value of learning lifespan-guided tool use strategies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 42 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

11 of 42 references.

References · 11

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Policy Distillationcited by 440
  3. 2019KETO: Learning Keypoint Representations for Tool Manipulationcited by 69
  4. 2023Creative Robot Tool Use with Large Language Modelscited by 43
  5. 2022Leveraging Language for Accelerated Learning of Tool Manipulationcited by 26
  6. 2021GIFT: Generalizable Interaction-aware Functional Tool Affordances without Labelscited by 19
  7. 2021How to select and use tools? : Active Perception of Target Objects Using Multimodal Deep Learningcited by 15
  8. 2022DefGraspSim: Physics-based simulation of grasp outcomes for 3D deformable objectscited by 10
  9. 2022Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomizationcited by 8
  10. 2022Understanding Physical Effects for Effective Tool-usecited by 7
  11. 2020A Reinforcement Learning Approach to Health Aware Control Strategycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
  2. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning
  3. 2026Survival Reinforcement Learning: Toward Scalable Self-Supervised RL
  4. 2026FARL: A Fully Automated Pipeline of RL Interface and Solver Design
  5. 2026Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection
  6. 2026ARM: Advantage Reward Modeling for Long-Horizon Manipulation
  7. 2025ToolRL: Reward is All Tool Learning Needs
  8. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning