mlsanityData to 14 Oct 2026

Tianshou: a Highly Modularized Deep Reinforcement Learning Library

arXiv 2107.14171 ↗First posted 29 Jul 2021cs.LGcited by 143

Not on the map. Why?

Cited by 51 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend. Tianshou intends to be research-friendly by providing a flexible and reliable infrastructure of DRL algorithms. It supports online and offline training with more than 20 classic algorithms through a unified interface. To facilitate related research and prove Tianshou's reliability, we have released Tianshou's benchmark of MuJoCo environments, covering eight classic algorithms with state-of-the-art performance. We open-sourced Tianshou at https://github.com/thu-ml/tianshou/.

Key facts. Cited by 143 papers in the map (data through 14 Oct 2026). Builds on 20 of its 55 references that are in the map. First posted on arXiv 29 Jul 2021.

Builds on

20 of 55 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2016Generative Adversarial Imitation Learningcited by 1,597
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  8. 2015Prioritized Experience Replaycited by 1,099
  9. 2018Learning Dexterous In-Hand Manipulationcited by 687
  10. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  11. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  12. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 121
8 more
  1. 2016Learning values across many orders of magnitudecited by 85
  2. 2017Time Limits in Reinforcement Learningcited by 79
  3. 2019ChainerRL: A Deep Reinforcement Learning Librarycited by 51
  4. 2020MushroomRL: Simplifying Reinforcement Learning Researchcited by 50
  5. 2017Multi-step Reinforcement Learning: A Unifying Algorithmcited by 42
  6. 2018Accelerated Methods for Deep Reinforcement Learningcited by 40
  7. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  8. 2017Efficient Parallel Methods for Deep Reinforcement Learningcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 22; 2023: 34; 2024: 36; 2025: 30; 2026: 14.

143 papers.

Sort:

By area:

  1. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  2. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  3. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  4. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  5. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 174
  6. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  7. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 119
  8. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  9. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 80
  10. 2023RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmarkcited by 71
  11. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 62
  12. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
38 more
  1. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 59
  2. 2023Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learningcited by 51
  3. 2024Gymnasium: A Standard Interface for Reinforcement Learning Environmentscited by 46
  4. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 37
  5. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  6. 2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
  7. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  8. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 27
  9. 2024EgoGen: An Egocentric Synthetic Data Generatorcited by 27
  10. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 26
  11. 2021Human-Level Control through Directly-Trained Deep Spiking Q-Networkscited by 25
  12. 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
  13. 2024PufferLib: Making Reinforcement Learning Libraries and Environments Play Nicecited by 24
  14. 2021Recurrent Off-policy Baselines for Memory-based Continuous Controlcited by 21
  15. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  16. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learningcited by 17
  17. 2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
  18. 2023GUARD: A Safe Reinforcement Learning Benchmarkcited by 13
  19. 2025Reevaluating Policy Gradient Methods for Imperfect-Information Gamescited by 13
  20. 2022A Survey on Reinforcement Learning Methods in Character Animationcited by 12
  21. 2022Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARLcited by 12
  22. 2024More Efficient Randomized Exploration for Reinforcement Learning via Approximate Samplingcited by 12
  23. 2022ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Rewardcited by 11
  24. 2024Building Minimal and Reusable Causal State Abstractions for Reinforcement Learningcited by 11
  25. 2025Benchmarking Quantum Reinforcement Learningcited by 10
  26. 2022Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidationcited by 9
  27. 2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9
  28. 2023Pearl: A Production-ready Reinforcement Learning Agentcited by 9
  29. 2024EasyRL4Rec: An Easy-to-use Library for Reinforcement Learning Based Recommender Systemscited by 9
  30. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
  31. 2025MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimizationcited by 9
  32. 2022Interactive Visual Reasoning under Uncertaintycited by 8
  33. 2023Offline Reinforcement Learning for Optimizing Production Bidding Policiescited by 8
  34. 2024DTR-Bench: An in silico Environment and Benchmark Platform for Reinforcement Learning Based Dynamic Treatment Regimecited by 8
  35. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  36. 2024On the Performance Analysis of Momentum Method: A Frequency Domain Perspectivecited by 8
  37. 2022Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variablescited by 7
  38. 2023Neural Episodic Control with State Abstractioncited by 7

50 most cited of 143 All 143 (JSON)