# The Recurrent Reinforcement Learning Crypto Agent

arXiv: https://arxiv.org/abs/2201.04699 · HTML: https://mlsanity.com/paper/2201.04699/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 56 references that are in the map.
- First posted on arXiv 12 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [Cryptocurrency Portfolio Management with Deep Reinforcement Learning](https://mlsanity.com/paper/1612.01277/) (2016)
- [Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning](https://mlsanity.com/paper/2003.03051/) (2020)
- [Reinforcement Learning for Systematic FX Trading](https://mlsanity.com/paper/2110.04745/) (2021)

## Built on by (most cited first, 4 of 4)

- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Sequential asset ranking in nonstationary time series](https://mlsanity.com/paper/2202.12186/) (2022)
- [Feature-Rich Long-term Bitcoin Trading Assistant](https://mlsanity.com/paper/2209.12664/) (2022)
- [Artificial Intelligence in Equity and Crypto Markets: Progress, Profitability Evidence, and the Limits of Automated Investing](https://mlsanity.com/paper/2609.04917/) (2026)

## Abstract (arXiv)

We demonstrate a novel application of online transfer learning for a digital assets trading agent. This agent uses a powerful feature space representation in the form of an echo state network, the output of which is made available to a direct, recurrent reinforcement learning agent. The agent learns to trade the XBTUSD (Bitcoin versus US Dollars) perpetual swap derivatives contract on BitMEX on an intraday basis. By learning from the multiple sources of impact on the quadratic risk-adjusted utility that it seeks to maximise, the agent avoids excessive over-trading, captures a funding profit, and can predict the market's direction. Overall, our crypto agent realises a total return of 350%, net of transaction costs, over roughly five years, 71% of which is down to funding profit. The annualised information ratio that it achieves is 1.46.