Adaptive critic design with graph Laplacian for online learning control of nonlinear systems

作者：Lian, Chuanqiang; Xu, Xin^*; Zuo, Lei; Huang, Zhenhua

来源：International Journal of Adaptive Control and Signal Processing, 2014, 28(3-5): 290-304.

DOI：10.1002/acs.2344

• manifold learning
• reinforcement learning
• adaptive critic designs
• learning control
• graph Laplacian

In recent years, reinforcement learning (RL) and approximate dynamic programming (ADP) have been widely studied in the community of artificial intelligence and machine learning. As an important class of RL and ADP methods, adaptive critic designs (ACDs) with function approximation have been studied to realize online learning control of nonlinear dynamical systems. However, how to construct efficient feature representations for approximating value functions or policies is still a difficult problem. In this paper, ACDs with graph Laplacian (GL) are proposed by integrating manifold learning methods into feature representations of ACDs. An online learning control algorithm called graph Laplacian dual heuristic programming (GL-DHP) is presented, and its performance is analyzed both theoretically and empirically. Because of the nonlinear approximation ability of feature representation with GL, the GL-DHP method has much better performance than previous DHP methods with manually designed neural networks. Simulation results on learning control of a ball and plate system, which is a typical nonlinear dynamical system with continuous state and action spaces, demonstrate the effectiveness of the GL-DHP method.

• 出版日期2014-3
• 单位中国人民解放军国防科学技术大学