# Learning When to Act: Communication-Efficient Reinforcement Learning via Run-Time Assurance

arXiv: https://arxiv.org/abs/2605.12561 · HTML: https://mlsanity.com/paper/2605.12561/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation](https://mlsanity.com/paper/1908.08342/) (2019)
- [Dynamic Weights in Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/1809.07803/) (2018)
- [Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems](https://mlsanity.com/paper/2110.03506/) (2021)
- [Deep Reinforcement Learning for Event-Triggered Control](https://mlsanity.com/paper/1809.05152/) (2018)
- [Learning Event-triggered Control from Data through Joint Optimization](https://mlsanity.com/paper/2008.04712/) (2020)
- [When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL](https://mlsanity.com/paper/2406.01163/) (2024)
- [Runtime Safety Assurance Using Reinforcement Learning](https://mlsanity.com/paper/2010.10618/) (2020)
- [Machine Learning in Event-Triggered Control: Recent Advances and Open Issues](https://mlsanity.com/paper/2009.12783/) (2020)
- [InterQ: A DQN Framework for Optimal Intermittent Control](https://mlsanity.com/paper/2504.09035/) (2025)

## Built on by (most cited first, 1 of 1)

- [Inverting Self-Triggered Control: Adversarial Reinforcement Learning for Sparse Denial-of-Service Attacks](https://mlsanity.com/paper/2609.12016/) (2026)

## Abstract (arXiv)

Safe reinforcement learning (RL) typically asks textitwhat an agent should do. We ask textitwhen it needs to act, and show that a single policy can jointly learn control inputs and communication-efficient timing decisions under a pointwise Lyapunov safety shield. We focus on stabilization around a known equilibrium, where CARE-based LQR backups, Lyapunov certificates, and classical Lyapunov-STC are well defined, enabling clean comparison against analytical baselines. A run-time assurance (RTA) layer overrides the policy via a one-step-ahead Lyapunov prediction and a precomputed LQR backup, providing a strictly stronger guarantee than constrained MDP methods that enforce safety only in expectation. On an inverted pendulum, cart--pole, and planar quadrotor, the learned policy achieves 1.91×, 1.45×, and 3.51× higher mean inter-sample interval (MSI) than a Lyapunov-triggered baseline; a fixed LQR controller at the same average rate is unstable on all three plants, showing that adaptive timing, not a lower average rate, makes sparsity safe. A CARE-derived Lyapunov reward transfers across environments without redesign, with a single weight wc controlling the stability--communication tradeoff; ablations confirm the RTA shield is essential, with its removal reducing MSI by 1.27--1.84× and degrading state norms. A preference-conditioned extension recovers the full tradeoff frontier from one model at tfrac211 of training compute, and SAC experiments show the results are algorithm-agnostic across discrete and continuous domains. A 12-state 3D quadrotor case study extends the framework to higher-dimensional systems where classical STC is intractable, and robustness to ±30% mass variation and disturbances shows graceful degradation, with the RTA absorbing what the learned policy cannot.