A mean-variance optimization problem for discounted Markov decision processes

作者：Guo, Xianping; Ye, Liuer; Yin, George^*

来源：European Journal of Operational Research, 2012, 220(2): 423-429.

DOI：10.1016/j.ejor.2012.01.051

• Mean-variance criterion
• Finite continuous-time MDPs
• Discounted reward
• Policy iteration algorithm
• Efficient frontier

In this paper, we consider a mean-variance optimization problem for Markov decision processes (MDPs) over the set of (deterministic stationary) policies. Different from the usual formulation in MDPs, we aim to obtain the mean-variance optimal policy that minimizes the variance over a set of all policies with a given expected reward. For continuous-time MDPs with the discounted criterion and finite-state and action spaces, we prove that the mean-variance optimization problem can be transformed to an equivalent discounted optimization problem using the conditional expectation and Markov properties. Then, we show that a mean-variance optimal policy and the efficient frontier can be obtained by policy iteration methods with a finite number of iterations. We also address related issues such as a mutual fund theorem and illustrate our results with an example.

• 出版日期2012-7-16
• 单位暨南大学; 中山大学