mlsanityData to 14 Oct 2026

ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation

arXiv 2503.01052 ↗First posted 2 Mar 2025cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance, especially when working within a limited budget. In this work, we aim to offer a third-party data valuation approach that benefits both data providers and model developers. We introduce a linearized future influence kernel (LinFiK), which assesses the value of individual data samples in improving LLM performance during training. We further propose ALinFiK, a learning strategy to approximate LinFiK, enabling scalable data valuation. Our comprehensive evaluations demonstrate that this approach surpasses existing baselines in effectiveness and efficiency, demonstrating significant scalability advantages as LLM parameters increase.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 2 Mar 2025.

Builds on

33 of 54 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  8. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  10. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  11. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  12. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
21 more
  1. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  2. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  3. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 184
  4. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 150
  5. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  6. 2020The Early Phase of Neural Network Trainingcited by 115
  7. 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 104
  8. 2024MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Modelscited by 100
  9. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 94
  10. 2024How to Train Data-Efficient LLMscited by 93
  11. 2021Input Similarity from the Neural Network Perspectivecited by 79
  12. 2024A Survey on Data Selection for LLM Instruction Tuningcited by 77
  13. 2024LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewingcited by 62
  14. 2024Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuningcited by 57
  15. 2024Entropy Law: The Story Behind Data Compression and LLM Performancecited by 44
  16. 2020A Survey on Data Pricing: from Economics to Data Sciencecited by 39
  17. 2024A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregationcited by 23
  18. 2023Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Datacited by 21
  19. 2021Data Pricing in Machine Learning Pipelinescited by 18
  20. 2024Can LLMs Reason in the Wild with Programs?cited by 13
  21. 2024Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaborationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025All You Need is One: Capsule Prompt Tuning with a Single Vectorcited by 8
  2. 2025VTBench: Evaluating Visual Tokenizers for Autoregressive Image Generationcited by 6
  3. 2026Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inferencecited by 4
  4. 2025BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretrainingcited by 3
  5. 2025ClusterUCB: Efficient Gradient-Based Data Selection for Targeted Fine-Tuning of LLMscited by 2
  6. 2026Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearningcited by 2
  7. 2026Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuationcited by 1