mlsanityData to 14 Oct 2026

Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning

arXiv 2303.04833 ↗First posted 24 Feb 2023econ.GN cs.LG q-fin.ECcited by 4

Not on the map. Why?

Abstract (arXiv)

We study a heterogeneous agent macroeconomic model with an infinite number of households and firms competing in a labor market. Each household earns income and engages in consumption at each time step while aiming to maximize a concave utility subject to the underlying market conditions. The households aim to find the optimal saving strategy that maximizes their discounted cumulative utility given the market condition, while the firms determine the market conditions through maximizing corporate profit based on the household population behavior. The model captures a wide range of applications in macroeconomic studies, and we propose a data-driven reinforcement learning framework that finds the regularized competitive equilibrium of the model. The proposed algorithm enjoys theoretical guarantees in converging to the equilibrium of the market at a sub-linear rate.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 83 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

33 of 83 references.

References · 33

  1. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  2. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  3. 2016Input Convex Neural Networkscited by 414
  4. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  5. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 264
  6. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  7. 2019Provably Efficient Exploration in Policy Optimizationcited by 205
  8. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 178
  9. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 161
  10. 2021Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraintscited by 124
  11. 2018Optimal Control Via Neural Networks: A Convex Approachcited by 85
  12. 2020Entropy Regularization for Mean Field Games with Learningcited by 58
21 more
  1. 2020Multiple Descent: Design Your Own Generalization Curvecited by 45
  2. 2021A Primal-Dual Approach to Constrained Markov Decision Processescited by 42
  3. 2020The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalizationcited by 39
  4. 2021Variance-Aware Off-Policy Evaluation with Linear Function Approximationcited by 34
  5. 2023Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcited by 30
  6. 2022A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Banditscited by 28
  7. 2020More Data Can Expand the Generalization Gap Between Adversarially Robust and Standard Modelscited by 27
  8. 2022Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processescited by 26
  9. 2020A General Framework for Learning Mean-Field Gamescited by 25
  10. 2021Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximationcited by 23
  11. 2021Learning Stochastic Shortest Path with Linear Function Approximationcited by 23
  12. 2015A Computational Framework for Multivariate Convex Regression and its Variantscited by 16
  13. 2022Learn to Match with No Regret: Reinforcement Learning in Markov Matching Marketscited by 16
  14. 2021Convergence and Alignment of Gradient Descent with Random Backpropagation Weightscited by 14
  15. 2018On the Landscape of Synchronization Networks: A Perspective from Nonconvex Optimizationcited by 11
  16. 2021DeepHAM: A Global Solution Method for Heterogeneous Agent Models with Aggregate Shockscited by 11
  17. 2020Meta Learning in the Continuous Time Limitcited by 10
  18. 2022Data-Driven Mirror Descent with Input-Convex Neural Networkscited by 8
  19. 2023Asymmetric Certified Robustness via Feature-Convex Neural Networkscited by 8
  20. 2019Mean Field Equilibrium: Uniqueness, Existence, and Comparative Staticscited by 2
  21. 2022Cascaded Gaps: Towards Gap-Dependent Regret for Risk-Sensitive Reinforcement Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
  2. 2025Structural Reinforcement Learning for Heterogeneous Agent Macroeconomicscited by 2
  3. 2024Taming Equilibrium Bias in Risk-Sensitive Multi-Agent Reinforcement Learningcited by 0
  4. 2024Large-Scale Contextual Market Equilibrium Computation through Deep Learningcited by 0