mlsanityData to 14 Oct 2026

Chaos-based reinforcement learning with TD3

arXiv 2405.09086 ↗First posted 15 May 2024cs.LG cs.AI cs.NEcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Chaos-based reinforcement learning (CBRL) is a method in which the agent's internal chaotic dynamics drives exploration. However, the learning algorithms in CBRL have not been thoroughly developed in previous studies, nor have they incorporated recent advances in reinforcement learning. This study introduced Twin Delayed Deep Deterministic Policy Gradients (TD3), which is one of the state-of-the-art deep reinforcement learning algorithms that can treat deterministic and continuous action spaces, to CBRL. The validation results provide several insights. First, TD3 works as a learning algorithm for CBRL in a simple goal-reaching task. Second, CBRL agents with TD3 can autonomously suppress their exploratory behavior as learning progresses and resume exploration when the environment changes. Finally, examining the effect of the agent's chaoticity on learning shows that there exists a suitable range of chaos strength in the agent's model to flexibly switch between exploration and exploitation and adapt to environmental changes.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 84 references that are in the map. First posted on arXiv 15 May 2024.

Builds on

31 of 84 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2015Continuous control with deep reinforcement learningcited by 3,407
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  10. 2023A Survey of Large Language Modelscited by 2,576
  11. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  12. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
19 more
  1. 2016OpenAI Gymcited by 1,736
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  4. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  5. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 451
  6. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  7. 2018Distributed Prioritized Experience Replaycited by 269
  8. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  9. 2023On the Creativity of Large Language Modelscited by 57
  10. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 54
  11. 2019Reinforcement Learning with Convolutional Reservoir Computingcited by 11
  12. 2017Criticality in the brain: A synthesis of neurobiology, models and cognitioncited by 9
  13. 2022Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learningcited by 9
  14. 2020Model-Size Reduction for Reservoir Computing by Concatenating Internal States Through Timecited by 8
  15. 2020Deep Echo State Q-Network (DEQN) and Its Application in Dynamic Spectrum Sharing for 5G and Beyondcited by 7
  16. 2023Learning Reservoir Dynamics with Temporal Self-Modulationcited by 4
  17. 2017Deep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Frameworkcited by 3
  18. 2020Sensitivity - Local Index to Control Chaoticity or Gradient Globally -cited by 2
  19. 2022Deep Q-network using reservoir computing with multi-layered readoutcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Enhanced Evolutionary Multi-Objective Deep Reinforcement Learning for Reliable and Efficient Wireless Rechargeable Sensor Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
  2. 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
  3. 2025Dyna-Style Reinforcement Learning Modeling and Control of Non-linear Dynamics
  4. 2026Hybrid TD3: Overestimation Bias Analysis and Stable Policy Optimization for Hybrid Action Space
  5. 2025Control of a Twin Rotor using Twin Delayed Deep Deterministic Policy Gradient (TD3)
  6. 2024iQRL -- Implicitly Quantized Representations for Sample-efficient Reinforcement Learning
  7. 2025Trajectory Entropy Reinforcement Learning for Robust Robot Motor Skill Learning
  8. 2025Data-assimilated model-informed reinforcement learning