mlsanityData to 7 Oct 2026

A study of first-passage time minimization via Q-learning in heated gridworlds

arXiv 2110.02129 ↗First posted 5 Oct 2021math.OC cond-mat.stat-mech cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Optimization of first-passage times is required in applications ranging from nanobots navigation to market trading. In such settings, one often encounters unevenly distributed noise levels across the environment. We extensively study how a learning agent fares in 1- and 2- dimensional heated gridworlds with an uneven temperature distribution. The results show certain bias effects in agents trained via simple tabular Q-learning, SARSA, Expected SARSA and Double Q-learning. While high learning rate prevents exploration of regions with higher temperature, low enough rate increases the presence of agents in such regions. The discovered peculiarities and biases of temporal-difference-based reinforcement learning methods should be taken into account in real-world physical applications and agent design.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 61 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

20 of 61 references.

References · 20

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  3. 2019Machine Learning for Fluid Mechanicscited by 380
  4. 2017Adversarial Attacks on Neural Network Policiescited by 314
  5. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  6. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  7. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  8. 2018Reinforcement Learning with Perturbed Rewardscited by 76
  9. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  10. 2017Reinforcement Learning with a Corrupted Reward Channelcited by 68
  11. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
  12. 2017Online Reinforcement Learning in Stochastic Gamescited by 67
8 more
  1. 2020Reinforcement Learning with Combinatorial Actions: An Application to Vehicle Routingcited by 61
  2. 2019Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewardscited by 58
  3. 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
  4. 2017Flow Navigation by Smart Microswimmers via Reinforcement Learningcited by 32
  5. 2019Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learningcited by 21
  6. 2019Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansioncited by 13
  7. 2021Automatic Risk Adaptation in Distributional Reinforcement Learningcited by 5
  8. 2019Solving Markov Decision Processes with Reachability Characterization from Mean First Passage Timescited by 1

Built on by

Not cited yet.