mlsanityData to 14 Oct 2026

Data Acquisition: A New Frontier in Data-centric AI

arXiv 2311.13712 ↗First posted 22 Nov 2023cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

As Machine Learning (ML) systems continue to grow, the demand for relevant and comprehensive datasets becomes imperative. There is limited study on the challenges of data acquisition due to ad-hoc processes and lack of consistent methodologies. We first present an investigation of current data marketplaces, revealing lack of platforms offering detailed information about datasets, transparent pricing, standardized data formats. With the objective of inciting participation from the data-centric AI community, we then introduce the DAM challenge, a benchmark to model the interaction between the data providers and acquirers. The benchmark was released as a part of DataPerf. Our evaluation of the submitted strategies underlines the need for effective data acquisition strategies in ML.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

18 of 39 references.

References · 18

  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  2. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  3. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  4. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 281
  5. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 165
  6. 2019Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithmscited by 148
  7. 2022DataPerf: Benchmarks for Data-Centric AI Developmentcited by 90
  8. 2020A Distributional Framework for Data Valuationcited by 80
  9. 2018A Marketplace for Data: An Algorithmic Solutioncited by 76
  10. 2023OpenDataVal: a Unified Benchmark for Data Valuationcited by 55
  11. 2020A Survey on Data Pricing: from Economics to Data Sciencecited by 39
  12. 2020Efficient computation and analysis of distributional Shapley valuescited by 38
6 more
  1. 2018Model-based Pricing for Machine Learning in a Data Marketplacecited by 37
  2. 2021Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Trainingcited by 28
  3. 2021Data Pricing in Machine Learning Pipelinescited by 18
  4. 2023Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sourcescited by 16
  5. 2021Data Acquisition for Improving Machine Learning Modelscited by 7
  6. 2020Try Before You Buy: A practical data purchasing algorithm for real-world data marketplacescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 1; 2026: 3.

9 papers.

  1. 2024DAVED: Data Acquisition via Experimental Design for Data Marketscited by 6
  2. 2024Data Distribution Valuationcited by 5
  3. 20242D-OOB: Attributing Data Contribution Through Joint Valuation Frameworkcited by 4
  4. 2026A Sustainable AI Economy Needs Data Deals That Work for Generatorscited by 3
  5. 2024Data Measurements for Decentralized Data Marketscited by 2
  6. 2024Data-Centric AI in the Age of Large Language Modelscited by 2
  7. 2026GraphSeek: Next-Generation Graph Analytics with LLMscited by 1
  8. 2025What's the next frontier for Data-centric AI? Data Savvy Agentscited by 0
  9. 2026Convex Dataset Valuation for Post-Trainingcited by 0