mlsanityData to 7 Oct 2026

TorchRL: A data-driven decision-making library for PyTorch

arXiv 2306.00577 ↗First posted 1 Jun 2023cs.LG cs.AIcited by 59

Not on the map. Why?

Abstract (arXiv)

PyTorch has ascended as a premier machine learning framework, yet it lacks a native and comprehensive library for decision and control tasks suitable for large development teams dealing with complex real-world data and environments. To address this issue, we propose TorchRL, a generalistic control library for PyTorch that provides well-integrated, yet standalone components. We introduce a new and flexible PyTorch primitive, the TensorDict, which facilitates streamlined algorithm development across the many branches of Reinforcement Learning (RL) and control. We provide a detailed description of the building blocks and an extensive overview of the library across domains and tasks. Finally, we experimentally demonstrate its reliability and flexibility and show comparative benchmarks to demonstrate its computational efficiency. TorchRL fosters long-term support and is publicly available on GitHub for greater reproducibility and collaboration within the research community. The code is open-sourced on GitHub.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Builds on 75 of its 105 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

75 of 105 references.

References · 75

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  12. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
38 more
  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  8. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  9. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  11. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  13. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  14. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  15. 2019Solving Rubik's Cube with a Robot Handcited by 699
  16. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  17. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  18. 2019The StarCraft Multi-Agent Challengecited by 589
  19. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  20. 2014OpenML: networked science in machine learningcited by 523
  21. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  22. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  23. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  24. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  25. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  26. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  27. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  28. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  29. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  30. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  31. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  32. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  33. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  34. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  35. 2018Distributed Prioritized Experience Replaycited by 260
  36. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  37. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  38. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 16; 2025: 20; 2026: 15.

59 papers.

Sort:

By area:

  1. 2023RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmarkcited by 70
  2. 2023Generative Flow Networks as Entropy-Regularized RLcited by 60
  3. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 35
  4. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  5. 2023OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Controlcited by 28
  6. 2024No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOcited by 27
  7. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 19
  8. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learningcited by 17
  9. 2025GEM: A Gym for Agentic LLMscited by 17
  10. 2024ACEGEN: Reinforcement learning of generative chemical agents for drug discoverycited by 14
  11. 2024CREW: Facilitating Human-AI Teaming Researchcited by 11
  12. 2024Controlling Behavioral Diversity in Multi-Agent Reinforcement Learningcited by 10
38 more
  1. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  2. 2023Pearl: A Production-ready Reinforcement Learning Agentcited by 9
  3. 2025Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Expertscited by 7
  4. 2025AFABench: A Generic Framework for Benchmarking Active Feature Acquisitioncited by 7
  5. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  6. 2025Reinforcement learning with combinatorial actions for coupled restless banditscited by 5
  7. 2023RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Controlcited by 4
  8. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  9. 2024In Search for Architectures and Loss Functions in Multi-Objective Reinforcement Learningcited by 3
  10. 2024Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Samplingcited by 3
  11. 2024The impact of behavioral diversity in multi-agent reinforcement learningcited by 3
  12. 2025URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehiclescited by 3
  13. 2026Scalable Production Scheduling: Linear Complexity via Unified Homogeneous Graphscited by 3
  14. 2025RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehiclescited by 2
  15. 2025Reinforcement Learning Increases Wind Farm Power Production by Enabling Closed-Loop Collaborative Controlcited by 2
  16. 2026Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Controlcited by 2
  17. 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environmentscited by 2
  18. 2024End-to-End Reinforcement Learning of Curative Curtailment with Partial Measurement Availabilitycited by 1
  19. 2025Experience Replay with Random Reshufflingcited by 1
  20. 2025Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Corecited by 1
  21. 2025When Is Diversity Rewarded in Cooperative Multi-Agent Learning?cited by 1
  22. 2025Scaling Multi-Agent Environment Co-Design with Diffusion Modelscited by 1
  23. 2026IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarkingcited by 1
  24. 2026ADDA: a Modular Framework for Representing, Simulating and Assimilating Dynamics with End-to-end Differentiabilitycited by 1
  25. 2024Probabilistic Shapley Value Modeling and Inferencecited by 0
  26. 2024BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGOcited by 0
  27. 2024AExGym: Benchmarks and Environments for Adaptive Experimentationcited by 0
  28. 2024Enhancing Heterogeneous Multi-Agent Cooperation in Decentralized MARL via GNN-driven Intrinsic Rewardscited by 0
  29. 2024Dynamic Learning Rate for Deep Reinforcement Learning: A Bandit Approachcited by 0
  30. 2024Multi-Agent Environments for Vehicle Routing Problemscited by 0
  31. 2024Generalising Battery Control in Net-Zero Buildings via Personalised Federated RLcited by 0
  32. 2025CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zerocited by 0
  33. 2025ObjectRL: An Object-Oriented Reinforcement Learning Codebasecited by 0
  34. 2025SegDAC: Visual Generalization in Reinforcement Learning via Dynamic Object Tokenscited by 0
  35. 2025TDHook: A Lightweight Framework for Interpretabilitycited by 0
  36. 2025Reinforcement Learning-Based Optimization of CT Acquisition and Reconstruction Parameters Through Virtual Imaging Trialscited by 0
  37. 2025MagBotSim: Physics-Based Simulation and Reinforcement Learning Environments for Magnetic Roboticscited by 0
  38. 2026Latent Spherical Flow Policy for Reinforcement Learning with Combinatorial Actionscited by 0

50 most cited of 59 All 59 (JSON)