A GRADIENT DESCENT SARSA(lambda) ALGORITHM BASED ON THE ADAPTIVE REWARD-SHAPING MECHANISM

作者：Liu, Quan^*; Fu, QiMing; Xiao, Fei; Fu, YuChen

来源：Intelligent Automation and Soft Computing, 2013, 19(4): 599-612.

DOI：10.1080/10798587.2013.869119

• reinforcement learning
• Sarsa (lambda)
• gradient descent
• reward-shaping
• adaptive

Based on the adaptive reward-shaping mechanism, we propose a novel gradient descent (GD) Sarsa(lambda) algorithm to solve the problems of ill initial performance and low convergence speed in the reinforcement learning tasks with continuous state space. Adaptive normalized radial basis ANRBF) network is used to shape reward. The reward-shaping mechanism propagates model knowledge to the learner in the form of the additional reward signal so that the initial performance and convergence speed can be improved effectively. A function approximation algorithm named ANRBF-GD-Sarsa(lambda) is proposed based on the ANRBF network. The convergence of ANRBF-GD-Sarsa(lambda) is analyzed theoretically. Experiments are conducted to show the good initial performance and high convergence speed of the proposed algorithm.

• 出版日期2013
• 单位苏州大学