Automatic abstraction controller in reinforcement learning agent via automata

作者：Mousavi Seyed Sajad^*; Ghazanfari Behzad; Mozayani Nasser; Jahed Motlagh Mohammad Reza

来源：Applied Soft Computing, 2014, 25: 118-128.

DOI：10.1016/j.asoc.2014.08.071

• Reinforcement learning
• Hierarchical reinforcement learning
• Cluster Multi agent learning

Reinforcement learning (RL) for solving large and complex problems faces the curse of dimensions problem. To overcome this problem, frameworks based on the temporal abstraction have been presented; each having their advantages and disadvantages. This paper proposes a new method like the strategies introduced in the hierarchical abstract machines (HAMs) to create a high-level controller layer of reinforcement learning which uses options. The proposed framework considers a non-deterministic automata as a controller to make a more effective use of temporally extended actions and state space clustering. This method can be viewed as a bridge between option and HAM frameworks, which tries to suggest a new framework to decrease the disadvantage of both by creating connection structures between them and at the same time takes advantages of them. Experimental results on different test environments show significant efficiency of the proposed method.

• 出版日期2014-12