# Dataset2Vec: Learning Dataset Meta-Features

arXiv: https://arxiv.org/abs/1905.11063 · HTML: https://mlsanity.com/paper/1905.11063/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 30 of 38)

- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML](https://mlsanity.com/paper/2106.06257/) (2021)
- [What to Pre-Train on? Efficient Intermediate Task Selection](https://mlsanity.com/paper/2104.08247/) (2021)
- [TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications](https://mlsanity.com/paper/2311.02971/) (2023)
- [Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How](https://mlsanity.com/paper/2306.03828/) (2023)
- [MetaOOD: Automatic Selection of OOD Detection Models](https://mlsanity.com/paper/2410.03074/) (2024)
- [Zero-Shot AutoML with Pretrained Models](https://mlsanity.com/paper/2206.08476/) (2022)
- [ADGym: Design Choices for Deep Anomaly Detection](https://mlsanity.com/paper/2309.15376/) (2023)
- [Deep Ranking Ensembles for Hyperparameter Optimization](https://mlsanity.com/paper/2303.15212/) (2023)
- [Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training](https://mlsanity.com/paper/2506.10952/) (2025)
- [M3OOD: Automatic Selection of Multimodal OOD Detectors](https://mlsanity.com/paper/2508.11936/) (2025)
- [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://mlsanity.com/paper/2203.08399/) (2022)
- [On the Hyperparameter Loss Landscapes of Machine Learning Models: An Exploratory Study](https://mlsanity.com/paper/2311.14014/) (2023)
- [Learning Abstract Task Representations](https://mlsanity.com/paper/2101.07852/) (2021)
- [Hyperparameter Optimization with Differentiable Metafeatures](https://mlsanity.com/paper/2102.03776/) (2021)
- [Towards Automated Error Analysis: Learning to Characterize Errors](https://mlsanity.com/paper/2201.05017/) (2022)
- [Meta-Statistical Learning: Supervised Learning of Statistical Estimators](https://mlsanity.com/paper/2502.12088/) (2025)
- [Consolidated learning -- a domain-specific model-free optimization strategy with examples for XGBoost and MIMIC-IV](https://mlsanity.com/paper/2201.11815/) (2022)
- [Tabular Few-Shot Generalization Across Heterogeneous Feature Spaces](https://mlsanity.com/paper/2311.10051/) (2023)
- [OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?](https://mlsanity.com/paper/2410.02735/) (2024)
- [Selective Cross-Task Distillation](https://mlsanity.com/paper/2204.11526/) (2022)
- [SeFNet: Bridging Tabular Datasets with Semantic Feature Nets](https://mlsanity.com/paper/2306.11636/) (2023)
- [A General Recipe for Automated Machine Learning in Practice](https://mlsanity.com/paper/2308.15647/) (2023)
- [Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms](https://mlsanity.com/paper/2411.17499/) (2024)
- [Analytics Modelling over Multiple Datasets using Vector Embeddings](https://mlsanity.com/paper/2502.17060/) (2025)
- [From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models](https://mlsanity.com/paper/2609.06912/) (2026)
- [Improving Hyperparameter Optimization by Planning Ahead](https://mlsanity.com/paper/2110.08028/) (2021)
- [Hyperparameter Tuning MLPs for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2403.04477/) (2024)
- [Rethinking of Encoder-based Warm-start Methods in Hyperparameter Optimization](https://mlsanity.com/paper/2403.04720/) (2024)
- [Neural-Guided Equation Discovery](https://mlsanity.com/paper/2503.16953/) (2025)

## Abstract (arXiv)

Meta-learning, or learning to learn, is a machine learning approach that utilizes prior learning experiences to expedite the learning process on unseen tasks. As a data-driven approach, meta-learning requires meta-features that represent the primary learning tasks or datasets, and are estimated traditonally as engineered dataset statistics that require expert domain knowledge tailored for every meta-task. In this paper, first, we propose a meta-feature extractor called Dataset2Vec that combines the versatility of engineered dataset meta-features with the expressivity of meta-features learned by deep neural networks. Primary learning tasks or datasets are represented as hierarchical sets, i.e., as a set of sets, esp. as a set of predictor/target pairs, and then a DeepSet architecture is employed to regress meta-features on them. Second, we propose a novel auxiliary meta-learning task with abundant data called dataset similarity learning that aims to predict if two batches stem from the same dataset or different ones. In an experiment on a large-scale hyperparameter optimization task for 120 UCI datasets with varying schemas as a meta-learning task, we show that the meta-features of Dataset2Vec outperform the expert engineered meta-features and thus demonstrate the usefulness of learned meta-features for datasets with varying schemas for the first time.