# Recurrent Reinforcement Learning: A Hybrid Approach

arXiv: https://arxiv.org/abs/1509.03044 · HTML: https://mlsanity.com/paper/1509.03044/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2015.

## Built on by (most cited first, 13 of 13)

- [Chain of Thought Imitation with Procedure Cloning](https://mlsanity.com/paper/2205.10816/) (2022)
- [Decision S4: Efficient Sequence-Based RL via State Spaces Layers](https://mlsanity.com/paper/2306.05167/) (2023)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency](https://mlsanity.com/paper/2205.13476/) (2022)
- [Real-Time Recurrent Learning using Trace Units in Reinforcement Learning](https://mlsanity.com/paper/2409.01449/) (2024)
- [Rethinking Transformers in Solving POMDPs](https://mlsanity.com/paper/2405.17358/) (2024)
- [Approximate information state based convergence analysis of recurrent Q-learning](https://mlsanity.com/paper/2306.05991/) (2023)
- [A Deep Recurrent-Reinforcement Learning Method for Intelligent AutoScaling of Serverless Functions](https://mlsanity.com/paper/2308.05937/) (2023)
- [On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models](https://mlsanity.com/paper/2110.03054/) (2021)
- [Robust Recurrent Reinforcement Learning under Evolving Hidden Disturbances with Application to Rover Wheel Slip](https://mlsanity.com/paper/2307.15931/) (2023)
- [How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem](https://mlsanity.com/paper/2106.08849/) (2021)
- [Intelligent Hybrid Resource Allocation in MEC-assisted RAN Slicing Network](https://mlsanity.com/paper/2405.17436/) (2024)
- [Spatial-Aware Decision-Making with Ring Attractors in Reinforcement Learning Systems](https://mlsanity.com/paper/2410.03119/) (2024)

## Abstract (arXiv)

Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states. It is in general very challenging to construct and infer hidden states as they often depend on the agent's entire interaction history and may require substantial domain knowledge. In this work, we investigate a deep-learning approach to learning the representation of states in partially observable tasks, with minimal prior knowledge of the domain. In particular, we propose a new family of hybrid models that combines the strength of both supervised learning (SL) and reinforcement learning (RL), trained in a joint fashion: The SL component can be a recurrent neural networks (RNN) or its long short-term memory (LSTM) version, which is equipped with the desired property of being able to capture long-term dependency on history, thus providing an effective way of learning the representation of hidden states. The RL component is a deep Q-network (DQN) that learns to optimize the control for maximizing long-term rewards. Extensive experiments in a direct mailing campaign problem demonstrate the effectiveness and advantages of the proposed approach, which performs the best among a set of previous state-of-the-art methods.