# Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representation arXiv: https://arxiv.org/abs/2311.13184 ยท HTML: https://mlsanity.com/paper/2311.13184/ ## Key facts - Cited by 22 papers in the map (data through 14 Oct 2026). - Builds on 16 of its 45 references that are in the map. - First posted on arXiv 22 Nov 2023. ## Builds on (most cited first, 16 of 16) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016) - [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020) - [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020) - [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022) - [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017) - [Automated Algorithm Selection: Survey and Perspectives](https://mlsanity.com/paper/1811.11597/) (2018) - [ASlib: A Benchmark Library for Algorithm Selection](https://mlsanity.com/paper/1506.02465/) (2015) - [AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks](https://mlsanity.com/paper/2306.08107/) (2023) - [Probabilistic Matrix Factorization for Automated Machine Learning](https://mlsanity.com/paper/1705.05355/) (2017) - [AutoField: Automating Feature Selection in Deep Recommender Systems](https://mlsanity.com/paper/2204.09078/) (2022) - [Automated Algorithm Selection: from Feature-Based to Feature-Free Approaches](https://mlsanity.com/paper/2203.13392/) (2022) - [Extreme Algorithm Selection With Dyadic Feature Representation](https://mlsanity.com/paper/2001.10741/) (2020) - [Explorative Data Analysis of Time Series based AlgorithmFeatures of CMA-ES Variants](https://mlsanity.com/paper/2104.08098/) (2021) - [HARRIS: Hybrid Ranking and Regression Forests for Algorithm Selection](https://mlsanity.com/paper/2210.17341/) (2022) ## Built on by (most cited first, 22 of 22) - [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024) - [LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation](https://mlsanity.com/paper/2403.01131/) (2024) - [Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization](https://mlsanity.com/paper/2411.00625/) (2024) - [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025) - [MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization](https://mlsanity.com/paper/2505.17745/) (2025) - [HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models](https://mlsanity.com/paper/2409.18893/) (2024) - [Neural Solver Selection for Combinatorial Optimization](https://mlsanity.com/paper/2410.09693/) (2024) - [Large Language Model-Driven Surrogate-Assisted Evolutionary Algorithm for Expensive Optimization](https://mlsanity.com/paper/2507.02892/) (2025) - [Algorithm Selection for Recommender Systems via Meta-Learning on Algorithm Characteristics](https://mlsanity.com/paper/2508.04419/) (2025) - [Learn to Relax with Large Language Models: Solving Constraint Optimization Problems via Bidirectional Coevolution](https://mlsanity.com/paper/2509.12643/) (2025) - [BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design](https://mlsanity.com/paper/2604.12898/) (2026) - [Meta-Black-Box Optimization with Ensemble Surrogate Modeling for Robustness-Accuracy Trade-off within SAEA](https://mlsanity.com/paper/2606.00862/) (2026) - [Online Learning with LLM Experts from Limited Feedback](https://mlsanity.com/paper/2609.05820/) (2026) - [Instance-Aware Algorithm Selection for Maximum Clique via a Dual-Channel Graph Neural Architecture](https://mlsanity.com/paper/2508.08005/) (2025) - [Intelligent Algorithm Selection for Recommender Systems: Meta-Learning via in-depth algorithm feature engineering](https://mlsanity.com/paper/2509.20134/) (2025) - [Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning](https://mlsanity.com/paper/2602.00540/) (2026) - [Towards automated data analysis: A guided framework for LLM-based risk estimation](https://mlsanity.com/paper/2603.04631/) (2026) - [Algorithm Selection with Zero Domain Knowledge via Text Embeddings](https://mlsanity.com/paper/2604.19753/) (2026) - [Relation Reasoning with LLMs in Expensive Optimization](https://mlsanity.com/paper/2605.02933/) (2026) - [Large Language Model for Operations Research Formulation Selection in Multi-Warehouse Inventory Allocation](https://mlsanity.com/paper/2607.25956/) (2026) - [MOSAIC: Adversarial Co-evolution of Specialist Heuristics and Problem Instances for LLM-based Automated Heuristic Design](https://mlsanity.com/paper/2608.07544/) (2026) - [Synthesizing Feature Extractors: An Agentic Approach for Algorithm Selection](https://mlsanity.com/paper/2608.17170/) (2026) ## Abstract (arXiv) Algorithm selection, a critical process of automated machine learning, aims to identify the most suitable algorithm for solving a specific problem prior to execution. Mainstream algorithm selection techniques heavily rely on problem features, while the role of algorithm features remains largely unexplored. Due to the intrinsic complexity of algorithms, effective methods for universally extracting algorithm information are lacking. This paper takes a significant step towards bridging this gap by introducing Large Language Models (LLMs) into algorithm selection for the first time. By comprehending the code text, LLM not only captures the structural and semantic aspects of the algorithm, but also demonstrates contextual awareness and library function understanding. The high-dimensional algorithm representation extracted by LLM, after undergoing a feature selection module, is combined with the problem representation and passed to the similarity calculation module. The selected algorithm is determined by the matching degree between a given problem and different algorithms. Extensive experiments validate the performance superiority of the proposed model and the efficacy of each key module. Furthermore, we present a theoretical upper bound on model complexity, showcasing the influence of algorithm representation and feature selection modules. This provides valuable theoretical guidance for the practical implementation of our method.