# Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methods

arXiv: https://arxiv.org/abs/1906.03139 · HTML: https://mlsanity.com/paper/1906.03139/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 8 of 8)

- [Top-KAST: Top-K Always Sparse Training](https://mlsanity.com/paper/2106.03517/) (2021)
- [EvoTorch: Scalable Evolutionary Computation in Python](https://mlsanity.com/paper/2302.12600/) (2023)
- [Learning Discrete Structured Variational Auto-Encoder using Natural Evolution Strategies](https://mlsanity.com/paper/2205.01324/) (2022)
- [ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces](https://mlsanity.com/paper/2101.07415/) (2021)
- [Natural Evolution Strategies as a Black Box Estimator for Stochastic Variational Inference](https://mlsanity.com/paper/2308.08053/) (2023)
- [AI planning in the imagination: High-level planning on learned abstract search spaces](https://mlsanity.com/paper/2308.08693/) (2023)
- [AlphaZeroES: Direct score maximization outperforms planning loss minimization](https://mlsanity.com/paper/2406.08687/) (2024)
- [Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule](https://mlsanity.com/paper/2606.29119/) (2026)

## Abstract (arXiv)

In this work we show that Evolution Strategies (ES) are a viable method for learning non-differentiable parameters of large supervised models. ES are black-box optimization algorithms that estimate distributions of model parameters; however they have only been used for relatively small problems so far. We show that it is possible to scale ES to more complex tasks and models with millions of parameters. While using ES for differentiable parameters is computationally impractical (although possible), we show that a hybrid approach is practically feasible in the case where the model has both differentiable and non-differentiable parameters. In this approach we use standard gradient-based methods for learning differentiable weights, while using ES for learning non-differentiable parameters - in our case sparsity masks of the weights. This proposed method is surprisingly competitive, and when parallelized over multiple devices has only negligible training time overhead compared to training with gradient descent. Additionally, this method allows to train sparse models from the first training step, so they can be much larger than when using methods that require training dense models first. We present results and analysis of supervised feed-forward models (such as MNIST and CIFAR-10 classification), as well as recurrent models, such as SparseWaveRNN for text-to-speech.