# Blind Decision Making: Reinforcement Learning with Delayed Observations

arXiv: https://arxiv.org/abs/2011.07715 · HTML: https://mlsanity.com/paper/2011.07715/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delays](https://mlsanity.com/paper/2108.07555/) (2021)
- [Delayed Reinforcement Learning by Imitation](https://mlsanity.com/paper/2205.05569/) (2022)
- [Neural Laplace Control for Continuous-time Delayed Systems](https://mlsanity.com/paper/2302.12604/) (2023)
- [DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays](https://mlsanity.com/paper/2406.03102/) (2024)
- [Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference](https://mlsanity.com/paper/2412.14355/) (2024)
- [Efficient Reinforcement Learning with Impaired Observability: Learning to Act with Delayed and Missing State Observations](https://mlsanity.com/paper/2306.01243/) (2023)
- [Posterior Sampling with Delayed Feedback for Reinforcement Learning with Linear Function Approximation](https://mlsanity.com/paper/2310.18919/) (2023)
- [Reinforcement Learning with Delayed, Composite, and Partially Anonymous Reward](https://mlsanity.com/paper/2305.02527/) (2023)
- [Missing Data Multiple Imputation for Tabular Q-Learning in Online RL](https://mlsanity.com/paper/2510.10709/) (2025)
- [Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Survey](https://mlsanity.com/paper/2602.00399/) (2026)
- [Novel Actor-Critic Algorithm for Robust Decision Making of CAV under Delays and Loss of V2X Data](https://mlsanity.com/paper/2405.05072/) (2024)
- [Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning](https://mlsanity.com/paper/2408.05460/) (2024)
- [State-Action Inpainting Diffuser for Continuous Control with Delay](https://mlsanity.com/paper/2603.01553/) (2026)
- [Minimax Optimal Strategy for Delayed Observations in Online Reinforcement Learning](https://mlsanity.com/paper/2603.03480/) (2026)
- [Delay-Empowered Causal Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2605.12261/) (2026)

## Abstract (arXiv)

Reinforcement learning typically assumes that the state update from the previous actions happens instantaneously, and thus can be used for making future decisions. However, this may not always be true. When the state update is not available, the decision taken is partly in the blind since it cannot rely on the current state information. This paper proposes an approach, where the delay in the knowledge of the state can be used, and the decisions are made based on the available information which may not include the current state information. One approach could be to include the actions after the last-known state as a part of the state information, however, that leads to an increased state-space making the problem complex and slower in convergence. The proposed algorithm gives an alternate approach where the state space is not enlarged, as compared to the case when there is no delay in the state update. Evaluations on the basic RL environments further illustrate the improved performance of the proposed algorithm.