# Deep Q-Learning versus Proximal Policy Optimization: Performance Comparison in a Material Sorting Task

arXiv: https://arxiv.org/abs/2306.01451 · HTML: https://mlsanity.com/paper/2306.01451/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 5 of 5)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Truly Proximal Policy Optimization](https://mlsanity.com/paper/1903.07940/) (2019)
- [Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learning](https://mlsanity.com/paper/2103.04616/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Deep Recurrent-Reinforcement Learning Method for Intelligent AutoScaling of Serverless Functions](https://mlsanity.com/paper/2308.05937/) (2023)
- [Reconfiguration of pivoting cube ensembles under local sensing constraints using geometric deep learning](https://mlsanity.com/paper/2509.03140/) (2025)

## Abstract (arXiv)

This paper presents a comparison between two well-known deep Reinforcement Learning (RL) algorithms: Deep Q-Learning (DQN) and Proximal Policy Optimization (PPO) in a simulated production system. We utilize a Petri Net (PN)-based simulation environment, which was previously proposed in related work. The performance of the two algorithms is compared based on several evaluation metrics, including average percentage of correctly assembled and sorted products, average episode length, and percentage of successful episodes. The results show that PPO outperforms DQN in terms of all evaluation metrics. The study highlights the advantages of policy-based algorithms in problems with high-dimensional state and action spaces. The study contributes to the field of deep RL in context of production systems by providing insights into the effectiveness of different algorithms and their suitability for different tasks.