# LExCI: A Framework for Reinforcement Learning with Embedded Systems

arXiv: https://arxiv.org/abs/2312.02739 · HTML: https://mlsanity.com/paper/2312.02739/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [Controlling an Inverted Pendulum with Policy Gradient Methods-A Tutorial](https://mlsanity.com/paper/2105.07998/) (2021)
- [A Deep Reinforcement Learning Approach towards Pendulum Swing-up Problem based on TF-Agents](https://mlsanity.com/paper/2106.09556/) (2021)
- [Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop](https://mlsanity.com/paper/2310.17671/) (2023)

## Built on by (most cited first, 3 of 3)

- [Safe Reinforcement Learning for Real-World Engine Control](https://mlsanity.com/paper/2501.16613/) (2025)
- [Hybrid Reinforcement Learning and Model Predictive Control for Adaptive Control of Hydrogen-Diesel Dual-Fuel Combustion](https://mlsanity.com/paper/2504.16875/) (2025)
- [Production-Ready Automated ECU Calibration using Residual Reinforcement Learning](https://mlsanity.com/paper/2604.07059/) (2026)

## Abstract (arXiv)

Advances in artificial intelligence (AI) have led to its application in many areas of everyday life. In the context of control engineering, reinforcement learning (RL) represents a particularly promising approach as it is centred around the idea of allowing an agent to freely interact with its environment to find an optimal strategy. One of the challenges professionals face when training and deploying RL agents is that the latter often have to run on dedicated embedded devices. This could be to integrate them into an existing toolchain or to satisfy certain performance criteria like real-time constraints. Conventional RL libraries, however, cannot be easily utilised in conjunction with that kind of hardware. In this paper, we present a framework named LExCI, the Learning and Experiencing Cycle Interface, which bridges this gap and provides end-users with a free and open-source tool for training agents on embedded systems using the open-source library RLlib. Its operability is demonstrated with two state-of-the-art RL-algorithms and a rapid control prototyping system.