A simulation-based approach to stochastic dynamic programming

作者：Polson Nicholas G^*; Sorensen Morten

来源：Applied Stochastic Models in Business and Industry, 2011, 27(2): 151-163.

DOI：10.1002/asmb.896

• Q-learning
• Bellman
• dynamic programming
• sequential Monte Carlo
• stochastic simulation
• simulated annealing

In this paper we develop a simulation-based approach to stochastic dynamic programming. To solve the Bellman equation we construct Monte Carlo estimates of Q-values. Our method is scalable to high dimensions and works in both continuous and discrete state and decision spaces while avoiding discretization errors that plague traditional methods. We provide a geometric convergence rate. We illustrate our methodology with a dynamic stochastic investment problem.

• 出版日期2011-4