mlsanityData to 7 Oct 2026

Adaptive Optimization for Enhanced Efficiency in Large-Scale Language Model Training

arXiv 2412.04718 ↗First posted 6 Dec 2024cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

With the rapid development of natural language processing technology, large-scale language models (LLM) have achieved remarkable results in a variety of tasks. However, how to effectively train these huge models and improve their performance and computational efficiency remains an important challenge. This paper proposes an improved method based on adaptive optimization algorithm, aiming to improve the training efficiency and final performance of LLM. Through comparative experiments on the SQuAD and GLUE data sets, the experimental results show that compared with traditional optimization algorithms (such as SGD, Momentum, AdaGrad, RMSProp and Adam), the adaptive optimization algorithm we proposed has better accuracy and F1 score. Both have achieved significant improvements, especially showed stronger training capabilities when processed large-scale texts and complex tasks. The research results verify the advantages of adaptive optimization algorithms in large-scale language model training and provide new ideas and directions for future optimization methods.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 13 of its 25 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

13 of 25 references.

References · 13

  1. 2024Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Miningcited by 47
  2. 2024Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networkscited by 33
  3. 2024Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generationcited by 17
  4. 2024Emotion-Aware Interaction Design in Intelligent User Interface Using Multi-Modal Deep Learningcited by 13
  5. 2024Metric Learning for Tag Recommendation: Tackling Data Sparsity and Cold Start Issuescited by 12
  6. 2024Self-Supervised Learning in Deep Networks: A Pathway to Robust Few-Shot Classificationcited by 11
  7. 2024LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuningcited by 10
  8. 2024Adaptive Cache Management for Complex Storage Systems Using CNN-LSTM-Based Spatiotemporal Predictioncited by 9
  9. 2024Graph Neural Network-Based Entity Extraction and Relationship Reasoning in Complex Knowledge Graphscited by 9
  10. 2024Optimizing Gesture Recognition for Seamless UI Interaction Using Convolutional Neural Networkscited by 9
  11. 2024Enhancing Few-Shot Learning with Integrated Data and GAN Model Approachescited by 9
  12. 2024Deep Learning for Medical Text Processing: BERT Model Fine-Tuning and Comparative Studycited by 8
1 more
  1. 2024Deep Learning with HM-VGG: AI Strategies for Multi-modal Image Analysiscited by 7

Built on by

Cited by year

Citing papers per year: 2024: 6; 2025: 7.

13 papers.

Sort:

By area:

  1. 2024Machine Learning Techniques for Pattern Recognition in High-Dimensional Data Miningcited by 8
  2. 2024AI-Driven Health Monitoring of Distributed Computing Architecture: Insights from XGBoost and SHAPcited by 7
  3. 2024Dynamic Scheduling Strategies for Resource Optimization in Computing Environmentscited by 6
  4. 2025Selective Attention Federated Learning: Improving Privacy and Efficiency for Clinical Text Classificationcited by 5
  5. 2025Physics-Informed Neural Networks for Enhanced Interface Preservation in Lattice Boltzmann Multiphase Simulationscited by 4
  6. 2024Computer Vision-Driven Gesture Recognition: Toward Natural and Intuitive Human-Computercited by 3
  7. 2024Deep Learning in Image Classification: Evaluating VGG19's Performance on Complex Visual Datacited by 3
  8. 2024Dynamic User Interface Generation for Enhanced Human-Computer Interaction Using Variational Autoencoderscited by 2
  9. 2025LBM-GNN: Graph Neural Network Enhanced Lattice Boltzmann Methodcited by 2
  10. 2025Agentic Metacognition: Designing a "Self-Aware" Low-Code Agent for Failure Prediction and Human Handoffcited by 2
  11. 2025Memory Management and Contextual Consistency for Long-Running Low-Code Agentscited by 2
  12. 2025Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generationcited by 2
1 more
  1. 2025Dynamic Policy Induction for Adaptive Prompt Optimization: Bridging the Efficiency-Accuracy Gap via Lightweight Reinforcement Learningcited by 0