Discovering Hidden Gems in Model Repositories
Not on the map. Why?
Abstract (arXiv)
Public repositories host millions of fine-tuned models, yet community usage remains disproportionately concentrated on a small number of foundation checkpoints. We investigate whether this concentration reflects efficient market selection or if superior models are systematically overlooked. Through an extensive evaluation of over 2,000 models, we show the prevalence of "hidden gems", unpopular fine-tunes that significantly outperform their popular counterparts. Notably, within the Llama-3.1-8B family, we find rarely downloaded checkpoints that improve math performance from 83.2% to 96.0% without increasing inference costs. However, discovering these models through exhaustive evaluation of every uploaded model is computationally infeasible. We therefore formulate model discovery as a Multi-Armed Bandit problem and accelerate the Sequential Halving search algorithm by using shared query sets and aggressive elimination schedules. Our method retrieves top models with as few as 50 queries per candidate, accelerating discovery by over 50x.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 52 of its 79 references that are in the map. First posted on arXiv 29 Jan 2026.
Builds on
52 of 79 references.
References · 52
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2018Model Cards for Model Reportingcited by 1,080
- 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
- 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 310
- 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
- 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
- 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
- 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 147
- 2019Transferability and Hardness of Supervised Classification Taskscited by 127
- 2024Model merging with SVD to tie the Knotscited by 122
38 more
- 2016Simple Bayesian Algorithms for Best Arm Identificationcited by 117
- 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
- 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 86
- 2020Classifying the classifier: dissecting the weight space of neural networkscited by 65
- 2024Learning to Route Among Specialized Experts for Zero-Shot Generalizationcited by 65
- 2022Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weightscited by 64
- 2024Graph Neural Networks for Learning Equivariant Representations of Neural Networkscited by 59
- 2023Knowledge is a Region in Weight Space for Fine-tuned Language Modelscited by 58
- 2023Graph Metanetworks for Processing Diverse Neural Architecturescited by 57
- 2022ColD Fusion: Collaborative Descent for Distributed Multitask Finetuningcited by 52
- 2023Rosetta Neurons: Mining the Common Units in a Model Zoocited by 46
- 2024Interpreting the Weight Space of Customized Diffusion Modelscited by 36
- 2025Symmetry in Neural Network Parameter Spacescited by 36
- 2023Model Spider: Learning to Rank Pre-Trained Models Efficientlycited by 32
- 2024Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on Hugging Facecited by 27
- 2025We Should Chart an Atlas of All the World's Modelscited by 25
- 2024Improved Generalization of Weight Space Networks via Augmentationscited by 24
- 2022Where to start? Analyzing the potential value of intermediate modelscited by 23
- 2025How to Select Datapoints for Efficient Human Evaluation of NLG Models?cited by 23
- 2024Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelscited by 22
- 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
- 2024Label-Efficient Model Selection for Text Generationcited by 19
- 2024PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarkscited by 19
- 2024Stylus: Automatic Adapter Selection for Diffusion Modelscited by 18
- 2024The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hubcited by 18
- 2024Deep Linear Probe Generators for Weight Space Learningcited by 17
- 2025Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViTcited by 17
- 2024Recovering the Pre-Fine-Tuning Weights of Generative Modelscited by 16
- 2024Learning Useful Representations of Recurrent Neural Network Weight Matricescited by 15
- 2022Bayesian Fixed-Budget Best-Arm Identificationcited by 13
- 2025Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributionscited by 12
- 2021GAN Cocktail: mixing GANs without dataset accesscited by 8
- 2024Learning on Model Weights using Tree Expertscited by 7
- 2025AI Supply Chains: An Emerging Ecosystem of AI Actors, Products, and Servicescited by 7
- 2025Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weightscited by 5
- 2025Learning Model Representations Using Publicly Available Model Hubscited by 4
- 2025Economies of Open Intelligence: Tracing Power & Participation in the Model Ecosystemcited by 4
- 2025Data Swarms: Optimizable Generation of Synthetic Evaluation Datacited by 2
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.