Average cost criterion induced by the regular utility function for continuous-time Markov decision processes

作者：Wei, Qingda; Chen, Xian^*

来源：Discrete Event Dynamic Systems, 2017, 27(3): 501-524.

DOI：10.1007/s10626-017-0237-x

• Continuous-time Markov decision processes
• Regular utility function
• U-average cost criterion
• Optimal policy

In this paper we study the average cost criterion induced by the regular utility U-average cost criterion) for continuous-time Markov decision processes. This criterion is a generalization of the risk-sensitive average cost and expected average cost criteria. We first introduce an auxiliary risk-sensitive first passage optimization problem and obtain the properties of the corresponding optimal value function under the slight conditions. Then we show that the pair of the optimal value functions of the risk-sensitive average cost criterion and the risk-sensitive first passage criterion is a solution to the optimality equation of the risk-sensitive average cost criterion allowing the risk-sensitivity parameter to take any nonzero value. Moreover, we have that the optimal value function of the risk-sensitive average cost criterion is continuous with respect to the risk-sensitivity parameter. Finally, we give the connections between the U-average cost criterion and the average cost criteria induced by the identity function and the exponential utility function, and prove the existence of a U-average optimal deterministic stationary policy in the class of all randomized Markov policies.

• 出版日期2017-9
• 单位厦门大学; 华侨大学