# Data Valuation using Reinforcement Learning

arXiv: https://arxiv.org/abs/1909.11671 · HTML: https://mlsanity.com/paper/1909.11671/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2019.

## Built on by (most cited first, 30 of 116)

- [Data Banzhaf: A Robust Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2205.15466/) (2022)
- [DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models](https://mlsanity.com/paper/2310.00902/) (2023)
- [What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions](https://mlsanity.com/paper/2405.13954/) (2024)
- [Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.08566/) (2023)
- [Learning Fast Sample Re-weighting Without Reward Data](https://mlsanity.com/paper/2109.03216/) (2021)
- [LAVA: Data Valuation without Pre-Specified Learning Algorithms](https://mlsanity.com/paper/2305.00054/) (2023)
- [Data Curation Alone Can Stabilize In-context Learning](https://mlsanity.com/paper/2212.10378/) (2022)
- [Towards Fairness-Aware Federated Learning](https://mlsanity.com/paper/2111.01872/) (2021)
- [OpenDataVal: a Unified Benchmark for Data Valuation](https://mlsanity.com/paper/2306.10577/) (2023)
- [HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks](https://mlsanity.com/paper/2102.02515/) (2021)
- [Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value](https://mlsanity.com/paper/2304.07718/) (2023)
- [CS-Shapley: Class-wise Shapley Values for Data Valuation in Classification](https://mlsanity.com/paper/2211.06800/) (2022)
- [Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognition](https://mlsanity.com/paper/2206.04790/) (2022)
- [Improving Fairness for Data Valuation in Horizontal Federated Learning](https://mlsanity.com/paper/2109.09046/) (2021)
- [Understanding Instance-based Interpretability of Variational Auto-Encoders](https://mlsanity.com/paper/2105.14203/) (2021)
- [Weak-to-Strong Generalization Through the Data-Centric Lens](https://mlsanity.com/paper/2412.03881/) (2024)
- [Data Pricing in Machine Learning Pipelines](https://mlsanity.com/paper/2108.07915/) (2021)
- [Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling](https://mlsanity.com/paper/2410.03735/) (2024)
- [Data Valuation and Detections in Federated Learning](https://mlsanity.com/paper/2311.05304/) (2023)
- [Estimating informativeness of samples with Smooth Unique Information](https://mlsanity.com/paper/2101.06640/) (2021)
- [FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning](https://mlsanity.com/paper/2106.02310/) (2021)
- [DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems](https://mlsanity.com/paper/2211.05764/) (2022)
- [Synthetic Sample Selection for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2304.02846/) (2023)
- [Efficient Client Contribution Evaluation for Horizontal Federated Learning](https://mlsanity.com/paper/2102.13314/) (2021)
- [Deep transfer learning for image classification: a survey](https://mlsanity.com/paper/2205.09904/) (2022)
- [Synthetic Question Value Estimation for Domain Adaptation of Question Answering](https://mlsanity.com/paper/2203.08926/) (2022)
- [DU-Shapley: A Shapley Value Proxy for Efficient Dataset Valuation](https://mlsanity.com/paper/2306.02071/) (2023)
- [Adaptive Training Distributions with Scalable Online Bilevel Optimization](https://mlsanity.com/paper/2311.11973/) (2023)
- [Image quality assessment for machine learning tasks using meta-reinforcement learning](https://mlsanity.com/paper/2203.14258/) (2022)
- [Sample based Explanations via Generalized Representers](https://mlsanity.com/paper/2310.18526/) (2023)

## Abstract (arXiv)

Quantifying the value of data is a fundamental problem in machine learning. Data valuation has multiple important use cases: (1) building insights about the learning task, (2) domain adaptation, (3) corrupted sample discovery, and (4) robust learning. To adaptively learn data values jointly with the target task predictor model, we propose a meta learning framework which we name Data Valuation using Reinforcement Learning (DVRL). We employ a data value estimator (modeled by a deep neural network) to learn how likely each datum is used in training of the predictor model. We train the data value estimator using a reinforcement signal of the reward obtained on a small validation set that reflects performance on the target task. We demonstrate that DVRL yields superior data value estimates compared to alternative methods across different types of datasets and in a diverse set of application scenarios. The corrupted sample discovery performance of DVRL is close to optimal in many regimes (i.e. as if the noisy samples were known apriori), and for domain adaptation and robust learning DVRL significantly outperforms state-of-the-art by 14.6% and 10.8%, respectively.