Autoencoder, low rank approximation and pseudoinverse learning algorithm

作者：Wang, Ke; Guo, Ping; Xin, Xin; Ye, Zebin

来源：2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, 2017-10-05 To 2017-10-08.

DOI：10.1109/SMC.2017.8122732

• Control parameters
• Fully automated
• Gradient descent
• Learning control
• Low rank approximations
• Training efficiency
• Training process
• Trial and error

Deep multi-layer neural networks are generally trained using variants of the gradient descent based algorithm. However, this kind of algorithms usually encounter a series of shortcomings, such as low training efficiency, local minimum, difficult control parameter tuning, and gradient vanishing or exploding. Besides, for a specific application, how to design the structure of the network, that is, how many neurons in each hidden layer and how many hidden layers is needed, is also a very tricky problem and is usually solved by trial and error in practice. To overcome the shortcomings mentioned above, we present a fast and fully automated method to train stacked autoencoders based deep neural networks in this paper. The proposed method trains the stacked autoencoders adopting the pseudoinverse learning algorithm with the low rank approximation. The entire training process neither need to set the learning control parameters, nor specify the number of hidden layers and the number of neurons in each hidden layer. The experimental results show that the proposed method can achieve a comprehensively better performance in terms of training efficiency and accuracy.

• 出版日期2017
• 单位北京理工大学