# TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow

arXiv: https://arxiv.org/abs/1709.02878 · HTML: https://mlsanity.com/paper/1709.02878/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2017.

## Built on by (most cited first, 9 of 9)

- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Autonomous Reinforcement Learning via Subgoal Curricula](https://mlsanity.com/paper/2107.12931/) (2021)
- [Reinforcement Learning for Machine Learning Engineering Agents](https://mlsanity.com/paper/2509.01684/) (2025)
- [A Review of Reinforcement Learning in Financial Applications](https://mlsanity.com/paper/2411.12746/) (2024)
- [SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores](https://mlsanity.com/paper/2306.16688/) (2023)
- [RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation](https://mlsanity.com/paper/2312.14095/) (2023)
- [Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Games](https://mlsanity.com/paper/2209.10055/) (2022)
- [Discovering Adaptable Symbolic Algorithms from Scratch](https://mlsanity.com/paper/2307.16890/) (2023)
- [Project proposal: A modular reinforcement learning based automated theorem prover](https://mlsanity.com/paper/2209.02562/) (2022)

## Abstract (arXiv)

We introduce TensorFlow Agents, an efficient infrastructure paradigm for building parallel reinforcement learning algorithms in TensorFlow. We simulate multiple environments in parallel, and group them to perform the neural network computation on a batch rather than individual observations. This allows the TensorFlow execution engine to parallelize computation, without the need for manual synchronization. Environments are stepped in separate Python processes to progress them in parallel without interference of the global interpreter lock. As part of this project, we introduce BatchPPO, an efficient implementation of the proximal policy optimization algorithm. By open sourcing TensorFlow Agents, we hope to provide a flexible starting point for future projects that accelerates future research in the field.