mlsanityData to 14 Oct 2026

Sequential Policy Gradient for Adaptive Hyperparameter Optimization

arXiv 2506.15051 ↗First posted 18 Jun 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Reinforcement learning is essential for neural architecture search and hyperparameter optimization, but the conventional approaches impede widespread use due to prohibitive time and computational costs. Inspired by DeepSeek-V3 multi-token prediction architecture, we propose Sequential Policy Gradient modeling (SPG), a novel trajectory generation paradigm for lightweight online hyperparameter optimization. In contrast to conventional policy gradient methods, SPG extends the base model with temporary modules, enabling it to generate state-action (padded) trajectories in a single forward pass. Our experiments demonstrate that models gain performance when retrained with SPG on their original datasets and also outperform standard transfer fine-tuning. We evaluate on five datasets spanning computer vision (ImageNet, COCO), natural language processing (GLUE, SQuAD), and audio (SUPERB) to assess the industrial applicability of SPG. The proposed method demonstrates consistent improvements across widely adopted models, achieving performance gains of +0.2~7%, with significantly low computational costs. Fully reproducible code and pre-trained models: https://huggingface.co/UniversalAlgorithmic/SPG.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 41 of its 63 references that are in the map. First posted on arXiv 18 Jun 2025.

Builds on

41 of 63 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2017Proximal Policy Optimization Algorithmscited by 15,215
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  10. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  11. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  12. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
29 more
  1. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  8. 2015Trust Region Policy Optimizationcited by 2,580
  9. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  10. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  11. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  12. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  13. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  14. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  15. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  16. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  17. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  18. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  19. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  20. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  21. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  22. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  23. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  24. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  25. 2014Recurrent Models of Visual Attentioncited by 349
  26. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
  27. 2017BlockDrop: Dynamic Inference Paths in Residual Networkscited by 189
  28. 2015Conditional Computation in Neural Networks for faster modelscited by 167
  29. 2017Neural Optimizer Search with Reinforcement Learningcited by 82

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026AROpt: An Optimization Method for Autoregressive Time Series Forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Reparameterized Policy Learning for Multimodal Trajectory Optimization
  2. 2024Trajectory-Oriented Policy Optimization with Sparse Rewards
  3. 2025RLGS: Reinforcement Learning-Based Adaptive Hyperparameter Tuning for Gaussian Splatting
  4. 2026Delightful Policy Gradient
  5. 2026Accelerating Visual Policy Learning with Sampling-Based Model Predictive Control
  6. 2024Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers
  7. 2025Policy Gradient Guidance Enables Test Time Control
  8. 2026CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance