mlsanityData to 7 Oct 2026

Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective

arXiv 1901.11503 ↗First posted 31 Jan 2019cs.LG cs.AI cs.ROcited by 26

Not on the map. Why?

Abstract (arXiv)

Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space exploration methods and those in which they are superior. Through simple theoretical analyses, we prove that complexity of exploration in parameter space depends on the dimensionality of parameter space, while complexity of exploration in action space depends on both the dimensionality of action space and horizon length. This is also demonstrated empirically by comparing simple exploration methods on several model problems, including Contextual Bandit, Linear Regression and Reinforcement Learning in continuous control.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 5; 2022: 6; 2023: 2; 2024: 6; 2025: 5; 2026: 1.

26 papers.

Sort:

By area:

  1. 2024Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmarkcited by 113
  2. 2023DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Trainingcited by 76
  3. 2024Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuningcited by 53
  4. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 50
  5. 2021Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Designcited by 34
  6. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  7. 2020Average-reward model-free reinforcement learning: a systematic review and literature mappingcited by 19
  8. 2025Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuningcited by 17
  9. 2021Derivative-Free Reinforcement Learning: A Reviewcited by 15
  10. 2023Symmetry-Aware Robot Design with Structured Subgroupscited by 13
  11. 2022Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functionscited by 6
  12. 2024Fully Zeroth-Order Bilevel Programming via Gaussian Smoothingcited by 6
14 more
  1. 2022Assessing Quality-Diversity Neuro-Evolution Algorithms Performance in Hard Exploration Problemscited by 4
  2. 2022Learning One Abstract Bit at a Time Through Self-Invented Experiments Encoded as Neural Networkscited by 4
  3. 2024Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Methodcited by 4
  4. 2024Task2Morph: Differentiable Task-inspired Framework for Contact-Aware Robot Designcited by 4
  5. 2022Distributed Cooperative Multi-Agent Reinforcement Learning with Directed Coordination Graphcited by 2
  6. 2022Zero-Order One-Point Estimate with Distributed Stochastic Gradient-Tracking Techniquecited by 2
  7. 2024Forward Learning for Gradient-based Black-box Saliency Map Generationcited by 2
  8. 2025End-to-End On-Device Quantization-Aware Training for LLMs at Inference Costcited by 2
  9. 2022Black-Box Generalization: Stability of Zeroth-Order Learningcited by 1
  10. 2021Learning Sampling Policy for Faster Derivative Free Optimizationcited by 0
  11. 2021Model-Free Learning of Optimal Deterministic Resource Allocations in Wireless Systems via Action-Space Explorationcited by 0
  12. 2025Adaptive Policy Backbone via Shared Networkcited by 0
  13. 2025Bayesian Optimization for Dynamic Pricing and Learningcited by 0
  14. 2026ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Trainingcited by 0