Markov Decision Processes on Borel Spaces with Total Cost and Random Horizon

作者：Cruz Suarez Hugo^*; Ilhuicatzi Roldan Rocio; Montes de Oca Raul

来源：Journal of Optimization Theory and Applications, 2014, 162(1): 329-346.

DOI：10.1007/s10957-012-0262-8

• Markov decision process
• Total cost
• Random horizon
• Varying-time discount factor

This paper deals with Markov Decision Processes (MDPs) on Borel spaces with possibly unbounded costs. The criterion to be optimized is the expected total cost with a random horizon of infinite support. In this paper, it is observed that this performance criterion is equivalent to the expected total discounted cost with an infinite horizon and a varying-time discount factor. Then, the optimal value function and the optimal policy are characterized through some suitable versions of the Dynamic Programming Equation. Moreover, it is proved that the optimal value function of the optimal control problem with a random horizon can be bounded from above by the optimal value function of a discounted optimal control problem with a fixed discount factor. In this case, the discount factor is defined in an adequate way by the parameters introduced for the study of the optimal control problem with a random horizon. To illustrate the theory developed, a version of the Linear-Quadratic model with a random horizon and a Logarithm Consumption-Investment model are presented.

• 出版日期2014-7