mlsanityData to 7 Oct 2026

Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts

arXiv 2109.08857 ↗First posted 18 Sep 2021cs.NE cs.CV cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Evolutionary algorithms have been used in the digital art scene since the 1970s. A popular application of genetic algorithms is to optimize the procedural placement of vector graphic primitives to resemble a given painting. In recent years, deep learning-based approaches have also been proposed to generate procedural drawings, which can be optimized using gradient descent. In this work, we revisit the use of evolutionary algorithms for computational creativity. We find that modern evolution strategies (ES) algorithms, when tasked with the placement of shapes, offer large improvements in both quality and efficiency compared to traditional genetic algorithms, and even comparable to gradient-based methods. We demonstrate that ES is also well suited at optimizing the placement of shapes to fit the CLIP model, and can produce diverse, distinct geometric abstractions that are aligned with human interpretation of language. Videos and demo: https://es-clip.github.io/

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 22 of its 57 references that are in the map. First posted on arXiv 18 Sep 2021.

Builds on

22 of 57 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  4. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  5. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  6. 2017A Neural Representation of Sketch Drawingscited by 311
  7. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  8. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  9. 2020Differentiable Rendering: A Surveycited by 123
  10. 2019A Learned Representation for Scalable Vector Graphicscited by 91
  11. 2018Synthesizing Programs for Images using Reinforced Adversarial Learningcited by 72
  12. 2019Learning to Paint With Model-based Deep Reinforcement Learningcited by 70
10 more
  1. 2021Paint Transformer: Feed Forward Neural Painting with Stroke Predictioncited by 61
  2. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57
  3. 2019Neural Painters: A learned differentiable constraint for generating brushstroke paintingscited by 28
  4. 2019Unsupervised Doodling and Painting with Improved SPIRALcited by 21
  5. 2019Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applicationscited by 7
  6. 2020Scones: Towards Conversational Authoring of Sketchescited by 7
  7. 2020ClipUp: A Simple and Powerful Optimizer for Distribution-based Policy Evolutioncited by 7
  8. 2021Generative Art Using Neural Visual Grammars and Dual Encoderscited by 6
  9. 2019Shared Visual Abstractionscited by 4
  10. 2019Evolved Art with Transparent, Overlapping, and Geometric Shapescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 12; 2024: 5; 2025: 9; 2026: 6.

40 papers.

Sort:

By area:

  1. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  2. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  3. 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 75
  4. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 71
  5. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 63
  6. 2023Word-As-Image for Semantic Typographycited by 62
  7. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  8. 2022CLIPasso: Semantically-Aware Object Sketchingcited by 57
  9. 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
  10. 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
  11. 2022CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphicscited by 34
  12. 2022StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translationcited by 28
28 more
  1. 2024Automating the Search for Artificial Life with Foundation Modelscited by 20
  2. 2025NeuralSVG: An Implicit Representation for Text-to-Vector Generationcited by 16
  3. 2025SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generationcited by 16
  4. 2022CLIP-CLOP: CLIP-Guided Collage and Photomontagecited by 11
  5. 2023Prompt Evolution for Generative AI: A Classifier-Guided Approachcited by 9
  6. 2023Text-Guided Vector Graphics Customizationcited by 5
  7. 2022Stroke-based Rendering: From Heuristics to Deep Learningcited by 4
  8. 2023Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problemscited by 4
  9. 2023NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applicationscited by 4
  10. 2024Creative Text-to-Audio Generation via Synthesizer Programmingcited by 4
  11. 2023Text-guided Image-and-Shape Editing and Generation: A Short Surveycited by 3
  12. 2025WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstractioncited by 3
  13. 2022Bridging the prosody GAP: Genetic Algorithm with People to efficiently sample emotional prosodycited by 2
  14. 2026OmniLottie: Generating Vector Animations via Parameterized Lottie Tokenscited by 2
  15. 2023Evolving Three Dimension (3D) Abstract Art: Fitting Concepts by Languagecited by 1
  16. 2024SVGFusion: A VAE-Diffusion Transformer for Vector Graphic Generationcited by 1
  17. 2025Inference-Time Alignment of Diffusion Models via Evolutionary Algorithmscited by 1
  18. 2025Soft Quality-Diversity Optimizationcited by 1
  19. 2026Discount Model Search for Quality Diversity Optimization in High-Dimensional Measure Spacescited by 1
  20. 2026Hierarchical SVG Tokenization: Learning Compact Visual Programs for Scalable Vector Graphics Modelingcited by 1
  21. 2023Optimize and Reduce: A Top-Down Approach for Image Vectorizationcited by 0
  22. 2024Generative Visual Communication in the Era of Vision-Language Modelscited by 0
  23. 2025Solving Inverse Problems in Stochastic Self-Organizing Systems through Invariant Representationscited by 0
  24. 2025Participatory Evolution of Artificial Life Systems via Semantic Feedbackcited by 0
  25. 2025Evolve to Inspire: Novelty Search for Diverse Image Generationcited by 0
  26. 2026Quality-Diversity Optimization as Multi-Objective Optimizationcited by 0
  27. 2026Generating Physically Sound Designs from Text and a Set of Physical Constraintscited by 0
  28. 2026DiffBMP: Differentiable Rendering with Bitmap Primitivescited by 0