# The Differentiable Cross-Entropy Method

arXiv: https://arxiv.org/abs/1909.12830 · HTML: https://mlsanity.com/paper/1909.12830/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2019.

## Built on by (most cited first, 27 of 27)

- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraints](https://mlsanity.com/paper/2105.02343/) (2021)
- [MBRL-Lib: A Modular Library for Model-based Reinforcement Learning](https://mlsanity.com/paper/2104.10159/) (2021)
- [Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation](https://mlsanity.com/paper/2106.03273/) (2021)
- [Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2109.06619/) (2021)
- [SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems](https://mlsanity.com/paper/2210.12547/) (2022)
- [Scalable Online Planning via Reinforcement Learning Fine-Tuning](https://mlsanity.com/paper/2109.15316/) (2021)
- [Neuro-algorithmic Policies enable Fast Combinatorial Generalization](https://mlsanity.com/paper/2102.07456/) (2021)
- [Learning to Optimize in Model Predictive Control](https://mlsanity.com/paper/2212.02603/) (2022)
- [Learning Sampling Distributions for Model Predictive Control](https://mlsanity.com/paper/2212.02587/) (2022)
- [A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2310.06253/) (2023)
- [Graph Reinforcement Learning for Network Control via Bi-Level Optimization](https://mlsanity.com/paper/2305.09129/) (2023)
- [Planning with Learned Dynamic Model for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2108.02613/) (2021)
- [Deep Model Predictive Optimization](https://mlsanity.com/paper/2310.04590/) (2023)
- [Integrating Symmetry into Differentiable Planning with Steerable Convolutions](https://mlsanity.com/paper/2206.03674/) (2022)
- [Scaling up and Stabilizing Differentiable Planning with Implicit Differentiation](https://mlsanity.com/paper/2210.13542/) (2022)
- [A Simple Decentralized Cross-Entropy Method](https://mlsanity.com/paper/2212.08235/) (2022)
- [Decentralized Safe Multi-agent Stochastic Optimal Control using Deep FBSDEs and ADMM](https://mlsanity.com/paper/2202.10658/) (2022)
- [Preference-Optimized Pareto Set Learning for Blackbox Optimization](https://mlsanity.com/paper/2408.09976/) (2024)
- [SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradients](https://mlsanity.com/paper/2603.08824/) (2026)
- [Regret-Aware Black-Box Optimization with Natural Gradients, Trust-Regions and Entropy Control](https://mlsanity.com/paper/2206.06090/) (2022)
- [Bi-Level Reinforcement Learning Pathway for Sim-to-Real Optimality](https://mlsanity.com/paper/2510.17709/) (2025)
- [Zero-Sum Positional Differential Games as a Framework for Robust Reinforcement Learning: Deep Q-Learning Approach](https://mlsanity.com/paper/2405.02044/) (2024)
- [Skill-based Safe Reinforcement Learning with Risk Planning](https://mlsanity.com/paper/2505.01619/) (2025)
- [CORL: Reinforcement Learning of MILP Policies Solved via Branch and Bound](https://mlsanity.com/paper/2512.11169/) (2025)
- [Automated Random Embedding for Practical Bayesian Optimization with Unknown Effective Dimension](https://mlsanity.com/paper/2605.23473/) (2026)
- [Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT](https://mlsanity.com/paper/2610.03361/) (2026)

## Abstract (arXiv)

We study the cross-entropy method (CEM) for the non-convex optimization of a continuous and parameterized objective function and introduce a differentiable variant that enables us to differentiate the output of CEM with respect to the objective function's parameters. In the machine learning setting this brings CEM inside of the end-to-end learning pipeline where this has otherwise been impossible. We show applications in a synthetic energy-based structured prediction task and in non-convex continuous control. In the control setting we show how to embed optimal action sequences into a lower-dimensional space. DCEM enables us to fine-tune CEM-based controllers with policy optimization.