# Model Hubs and Beyond: Analyzing Model Popularity, Performance, and Documentation

arXiv: https://arxiv.org/abs/2503.15222 · HTML: https://mlsanity.com/paper/2503.15222/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets](https://mlsanity.com/paper/2310.04793/) (2023)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry](https://mlsanity.com/paper/2303.02552/) (2023)
- [Analyzing the Evolution and Maintenance of ML Models on Hugging Face](https://mlsanity.com/paper/2311.13380/) (2023)
- [Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trends](https://mlsanity.com/paper/2401.13177/) (2024)
- [Automatic Generation of Model and Data Cards: A Step Towards Responsible AI](https://mlsanity.com/paper/2405.06258/) (2024)
- [Unlocking Model Insights: A Dataset for Automated Model Card Generation](https://mlsanity.com/paper/2309.12616/) (2023)
- [Unveiling the Multi-Annotation Process: Examining the Influence of Annotation Quantity and Instance Difficulty on Model Performance](https://mlsanity.com/paper/2310.14572/) (2023)

## Built on by (most cited first, 4 of 4)

- [From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI Ecosystem](https://mlsanity.com/paper/2509.09873/) (2025)
- [Cost-Aware Model Orchestration for LLM-based Systems](https://mlsanity.com/paper/2512.01099/) (2025)
- [AgentHub: A Registry for Discoverable, Verifiable, and Reproducible AI Agents](https://mlsanity.com/paper/2510.03495/) (2025)
- [Bayesian Network Fusion of Large Language Models for Sentiment Analysis](https://mlsanity.com/paper/2510.26484/) (2025)

## Abstract (arXiv)

With the massive surge in ML models on platforms like Hugging Face, users often lose track and struggle to choose the best model for their downstream tasks, frequently relying on model popularity indicated by download counts, likes, or recency. We investigate whether this popularity aligns with actual model performance and how the comprehensiveness of model documentation correlates with both popularity and performance. In our study, we evaluated a comprehensive set of 500 Sentiment Analysis models on Hugging Face. This evaluation involved massive annotation efforts, with human annotators completing nearly 80,000 annotations, alongside extensive model training and evaluation. Our findings reveal that model popularity does not necessarily correlate with performance. Additionally, we identify critical inconsistencies in model card reporting: approximately 80% of the models analyzed lack detailed information about the model, training, and evaluation processes. Furthermore, about 88% of model authors overstate their models' performance in the model cards. Based on our findings, we provide a checklist of guidelines for users to choose good models for downstream tasks.