mlsanityData to 14 Oct 2026

CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero

arXiv 2505.21536 ↗First posted 24 May 2025cs.CY cs.CE cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

The demand of finite raw materials will keep increasing as they fuel modern society. Simultaneously, solutions for stopping carbon emissions in the short term are not available, thus making the net zero target extremely challenging to achieve at scale. The circular economy (CE) paradigm is gaining attention as a solution to address climate change and the uncertainties of supplies of critical materials. Hence, in this paper, we introduce CiRL, a deep reinforcement learning (DRL) library of environments focused on the circularity control of both solid and fluid materials. The integration of DRL into the design of material circularity is possible thanks to the formalism of thermodynamical material networks, which is underpinned by compartmental dynamical thermodynamics. Along with the focus on circularity, this library has three more features: the new CE-oriented environments are in the state-space form, which is typically used in dynamical systems analysis and control design; it is based on a state-of-the-art Python library of DRL algorithms, namely, Stable-Baselines3; and it is developed in Google Colaboratory to be accessible to researchers from different disciplines and backgrounds as is often the case for circular economy researchers and engineers. CiRL is intended to be a tool to generate AI-driven actions for optimizing the circularity of supply-recovery chains and to be combined with human-driven decisions derived from material flow analysis (MFA) studies. CiRL is publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 27 of its 91 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

27 of 91 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2017Hindsight Experience Replaycited by 1,136
  7. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  8. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
  9. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  10. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 119
  11. 2020FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Financecited by 87
  12. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
15 more
  1. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  2. 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 59
  3. 2019ChainerRL: A Deep Reinforcement Learning Librarycited by 51
  4. 2022skrl: Modular and Flexible Library for Reinforcement Learningcited by 49
  5. 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 45
  6. 2024PufferLib: Making Reinforcement Learning Libraries and Environments Play Nicecited by 24
  7. 2023LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgerycited by 14
  8. 2024A Review of Nine Physics Engines for Reinforcement Learning Researchcited by 12
  9. 2021Thermodynamical Material Networks for Modeling, Planning, and Control of Circular Material Flowscited by 4
  10. 2022A Comparison of Reinforcement Learning Frameworks for Software Testing Taskscited by 3
  11. 2022Circularity of Thermodynamical Material Networks: Indicators, Examples, and Algorithmscited by 3
  12. 2024A Unification Between Deep-Learning Vision, Compartmental Dynamical Thermodynamics, and Robotic Manipulation for a Circular Economycited by 2
  13. 2024Deep multi-intentional inverse reinforcement learning for cognitive multi-function radar inverse cognitioncited by 2
  14. 2024Synchronized Object Detection for Autonomous Sorting, Mapping, and Quantification of Materials in Circular Healthcarecited by 1
  15. 2025Circular Microalgae-Based Carbon Control for Net Zerocited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025CIRO7.2: A Material Network with Circularity of -7.2 and Reinforcement-Learning-Controlled Robotic Disassembler ↗
  2. 2025Reinforcement Learning and Life Cycle Assessment for a Circular Economy -- Towards Progressive Computer Science
  3. 2026Automated co-design of high-performance thermodynamic cycles via graph-based hierarchical reinforcement learning
  4. 2025AI-driven materials design: a mini-review
  5. 2025Greener Deep Reinforcement Learning: Analysis of Energy and Carbon Efficiency Across Atari Benchmarks
  6. 2026Generative AI for material design: A mechanics perspective from burgers to matter
  7. 2024Reinforcement Learning for Sustainable Energy: A Survey
  8. 2026A perspective on fluid mechanical environments for challenges in reinforcement learning