mlsanityData to 7 Oct 2026

General Information Metrics for Improving AI Model Training Efficiency

arXiv 2501.02004 ↗First posted 2 Jan 2025cs.LG cs.AI cs.ITcited by 4

Not on the map. Why?

Abstract (arXiv)

To address the growing size of AI model training data and the lack of a universal data selection methodology-factors that significantly drive up training costs -- this paper presents the General Information Metrics Evaluation (GIME) method. GIME leverages general information metrics from Objective Information Theory (OIT), including volume, delay, scope, granularity, variety, duration, sampling rate, aggregation, coverage, distortion, and mismatch to optimize dataset selection for training purposes. Comprehensive experiments conducted across diverse domains, such as CTR Prediction, Civil Case Prediction, and Weather Forecasting, demonstrate that GIME effectively preserves model performance while substantially reducing both training time and costs. Additionally, applying GIME within the Judicial AI Program led to a remarkable 39.56% reduction in total model training expenses, underscoring its potential to support efficient and sustainable AI development.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 74 references that are in the map. First posted on arXiv 2 Jan 2025.

Builds on

32 of 74 references.

References · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2018Datasheets for Datasetscited by 1,371
  11. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  12. 2023Textbooks Are All You Needcited by 561
20 more
  1. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  2. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  3. 2024Large Language Models: A Surveycited by 506
  4. 2019Learning Loss for Active Learningcited by 335
  5. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  6. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  7. 2017Conditional Time Series Forecasting with Convolutional Neural Networkscited by 121
  8. 2023Generative AIcited by 119
  9. 2024MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Modelscited by 96
  10. 2022A Comparative Survey of Deep Active Learningcited by 88
  11. 2022A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challengescited by 50
  12. 2020Towards Automated Neural Interaction Discovery for Click-Through Rate Predictioncited by 37
  13. 2018modAL: A modular active learning framework for Pythoncited by 35
  14. 2024Harnessing Diversity for Important Data Selection in Pretraining Large Language Modelscited by 34
  15. 2022Click-Through Rate Prediction in Online Advertising: A Literature Reviewcited by 31
  16. 2022Data-Centric Artificial Intelligencecited by 31
  17. 2021A Data-Centric Approach for Training Deep Neural Networks with Less Datacited by 28
  18. 2023Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learningcited by 26
  19. 2023Federated Learning-Empowered AI-Generated Content in Wireless Networkscited by 26
  20. 2021Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning in the Real Court Settingcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Information Science Principles of Machine Learning: A Causal Chain Meta-Framework Based on Formalized Information Mappingcited by 2
  2. 2026The COTe score: A decomposable framework for evaluating Document Layout Analysis modelscited by 1
  3. 2025Information Physics of Intelligence: Unifying Logical Depth and Entropy under Thermodynamic Constraintscited by 0
  4. 2026Averaged Evaluation Masks Capability Trade-Offs: Multi-Source Calibration for High-Sparsity LLM Pruningcited by 0