# Dynamic Observation Policies in Observation Cost-Sensitive Reinforcement Learning

arXiv: https://arxiv.org/abs/2307.02620 · HTML: https://mlsanity.com/paper/2307.02620/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 26 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 9 of 9)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learning](https://mlsanity.com/paper/1702.06054/) (2017)
- [Active Measure Reinforcement Learning for Observation Cost Minimization](https://mlsanity.com/paper/2005.12697/) (2020)
- [Controlled Online Optimization Learning (COOL): Finding the ground state of spin Hamiltonians with reinforcement learning](https://mlsanity.com/paper/2003.00011/) (2020)
- [Scientific Discovery and the Cost of Measurement -- Balancing Information and Cost in Reinforcement Learning](https://mlsanity.com/paper/2112.07535/) (2021)
- [ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry](https://mlsanity.com/paper/2305.14177/) (2023)

## Built on by (most cited first, 3 of 3)

- [To Measure or Not: A Cost-Sensitive, Selective Measuring Environment for Agricultural Management Decisions with Reinforcement Learning](https://mlsanity.com/paper/2501.12823/) (2025)
- [Robust Remote Reinforcement Learning over Unreliable Communication Channels using Homomorphic State Encoding](https://mlsanity.com/paper/2508.07722/) (2025)
- [Learning to Wait: Synchronizing Agents with the Physical World](https://mlsanity.com/paper/2512.16262/) (2025)

## Abstract (arXiv)

Reinforcement learning (RL) has been shown to learn sophisticated control policies for complex tasks including games, robotics, heating and cooling systems and text generation. The action-perception cycle in RL, however, generally assumes that a measurement of the state of the environment is available at each time step without a cost. In applications such as materials design, deep-sea and planetary robot exploration and medicine, however, there can be a high cost associated with measuring, or even approximating, the state of the environment. In this paper, we survey the recently growing literature that adopts the perspective that an RL agent might not need, or even want, a costly measurement at each time step. Within this context, we propose the Deep Dynamic Multi-Step Observationless Agent (DMSOA), contrast it with the literature and empirically evaluate it on OpenAI gym and Atari Pong environments. Our results, show that DMSOA learns a better policy with fewer decision steps and measurements than the considered alternative from the literature.