A novel neural network training algorithm based on generalized objective function

作者：Xu Bao Chang^*; Luo Xiong Lin; Wang Jin Shan

来源：Journal of China University of Petroleum(Edition of Natural Science), 2009, 33(6): 95-99.

• Convergence speed
• Convergent rate
• High-order derivative
• Iterative algorithm
• Learning rates
• Neural network training
• Nonlinear functions
• Novel neural network
• Objective functions
• Output layer
• Recursive equations
• Second orders
• Simulation result
• Squared errors
• Training algorithms
• Training patterns
• Training procedures
• Weight matrices

A novel training algorithm was proposed to improve the learning rate and stability of the multi-layer feedforward neural networks. The generalized objective function was constructed by adding an auxiliary constraint term to the sum of the squared errors in the algorithm. The weight matrix of output layer was trained using the generalized objective function. The recursive equations for training the weight matrix of output layer were derived using Newton iterative algorithm without any simplification. The auxiliary constraint term involves the requirement for the smoothness of output which could improve the stability of the algorithm. The high-order derivative information of the neuron action function was used during the training procedure, so the algorithm had high convergence speed. In the end, the algorithm was used to learn training pattern of different nonlinear function. Simulation results show that the convergent rate and accuracy of the algorithm are better than those of the Karayiannis's second-order learning algorithm.

• 出版日期2009
• 单位中国石油大学（华东）