# Optimization of Discrete Parameters Using the Adaptive Gradient Method and Directed Evolution

arXiv: https://arxiv.org/abs/2401.06834 · HTML: https://mlsanity.com/paper/2401.06834/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 14 of 14)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Sparse Sinkhorn Attention](https://mlsanity.com/paper/2002.11296/) (2020)
- [Generating Multi-label Discrete Patient Records using Generative Adversarial Networks](https://mlsanity.com/paper/1703.06490/) (2017)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Hierarchical Multiscale Recurrent Neural Networks](https://mlsanity.com/paper/1609.01704/) (2016)
- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Features for the 0-1 knapsack problem based on inclusionwise maximal solutions](https://mlsanity.com/paper/2211.09665/) (2022)

## Abstract (arXiv)

The problem is considered of optimizing discrete parameters in the presence of constraints. We use the stochastic sigmoid with temperature and put forward the new adaptive gradient method CONGA. The search for an optimal solution is carried out by a population of individuals. Each of them varies according to gradients of the 'environment' and is characterized by two temperature parameters with different annealing schedules. Unadapted individuals die, and optimal ones interbreed, the result is directed evolutionary dynamics. The proposed method is illustrated using the well-known combinatorial problem for optimal packing of a backpack (0-1 KP).