# A Novel Hybrid Heuristic-Reinforcement Learning Optimization Approach for a Class of Railcar Shunting Problems

arXiv: https://arxiv.org/abs/2603.05579 · HTML: https://mlsanity.com/paper/2603.05579/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 36 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 2 of 2)

- [A Novel Dynamic Programming Approach to the Train Marshalling Problem](https://mlsanity.com/paper/1903.08364/) (2019)
- [Optimizing Railcar Movements to Create Outbound Trains in a Freight Railyard](https://mlsanity.com/paper/2505.06510/) (2025)

## Built on by (most cited first, 1 of 1)

- [Optimization of the Railcar Assignment Problem Using Zone-based Double Deep Reinforcement Learning](https://mlsanity.com/paper/2608.18442/) (2026)

## Abstract (arXiv)

Railcar shunting is a core planning task in freight railyards, where yard planners need to disassemble and reassemble groups of railcars to form outbound trains. Classification tracks with access from one side only can be considered as stack structures, where railcars are added and removed from only one end, leading to a last-in-first-out (LIFO) retrieval order. In contrast, two-sided tracks function like queue structures, allowing railcars to be added from one end and removed from the opposite end, following a first-in-first-out (FIFO) order. We consider a problem requiring assembly of multiple outbound trains using two locomotives in a railyard with two-sided classification track access. To address this combinatorially challenging problem class, we decompose the problem into two subproblems, each with one-sided classification track access and a locomotive on each side. We present a novel Hybrid Heuristic-Reinforcement Learning (HHRL) framework that integrates railway-specific heuristic solution approaches with a reinforcement learning method, specifically Q-learning. The proposed framework leverages methods to decrease the state-action space and guide exploration during reinforcement learning. The results of a series of numerical experiments demonstrate the efficiency and quality of the HHRL algorithm in both one-sided access, single-locomotive problems and two-sided access, two-locomotive problems.