mlsanityData to 14 Oct 2026

Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms

arXiv 2407.15283 ↗First posted 21 Jul 2024cs.LG cs.AI cs.ROcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Industry is moving toward autonomous, network-connected machines that detect and adapt to changing conditions, including hardware faults. Conventional fault-tolerant design duplicates hardware and reroutes control logic; reinforcement learning (RL) offers a learning-based alternative. This paper presents the first systematic comparison of two RL algorithms -- Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) -- for integrating fault tolerance into control. Beyond algorithm choice, we investigate four knowledge-transfer strategies: retaining or discarding model parameters, and retaining or discarding storage contents. Performance is evaluated in two Gymnasium environments: Ant-v5 and FetchReachDense-v3. Results show rapid, fault-specific recovery with clear trade-offs. In Ant-v5, retaining PPO's parameters boosts early returns and remains the safest choice across all faults, while retaining SAC's parameters yields mixed outcomes. SAC's early performance further depends on whether the replay buffer is retained: beneficial when prior experiences match current dynamics, but harmful when they diverge. In FetchReachDense-v3, discarding both PPO's and SAC's parameters was most effective under sensor corruption. Across tasks, both algorithms recover near-normal performance within minutes in low-dimensional settings and within days in high-dimensional settings, highlighting a clear trade-off between adaptation speed and asymptotic performance. These findings demonstrate that RL can deliver robust fault tolerance and offer practical guidelines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 55 references that are in the map. First posted on arXiv 21 Jul 2024.

Builds on

25 of 55 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2016OpenAI Gymcited by 1,736
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  10. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  12. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
13 more
  1. 2014Robots that can adapt like animalscited by 204
  2. 2018Reinforced Continual Learningcited by 186
  3. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
  4. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 121
  5. 2023A Definition of Continual Reinforcement Learningcited by 101
  6. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  7. 2021Safe Driving via Expert Guided Policy Optimizationcited by 31
  8. 2020Fast Adaptation via Policy-Dynamics Value Functionscited by 13
  9. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 8
  10. 2024Data-driven Machinery Fault Diagnosis: A Comprehensive Reviewcited by 7
  11. 2022Egocentric Visual Self-Modeling for Autonomous Robot Dynamics Prediction and Adaptationcited by 5
  12. 2019When Your Robot Breaks: Active Learning During Plant Failurecited by 1
  13. 2024Bio-Inspired Compensatory Strategies for Damage to Flapping Robotic Propulsorscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EvolveGen: Algorithmic Level Hardware Model Checking Benchmark Generation through Reinforcement Learning
  2. 2025A Study of the Efficacy of Generative Flow Networks for Robotics and Machine Fault-Adaptation
  3. 2026TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing
  4. 2026Mixture-of-Experts RL for Fault-Tolerant Legged Locomotion ↗
  5. 2024Towards Fault Tolerance in Multi-Agent Reinforcement Learning
  6. 2024Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field
  7. 2025Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards
  8. 2025RL-TIME: Reinforcement Learning-based Task Replication in Multicore Embedded Systems