Alleviating the problem of local minima in Backpropagation through competitive learning

作者：Dai Qun^*; Liu Ningzhong

来源：Neurocomputing, 2012, 94: 152-158.

DOI：10.1016/j.neucom.2012.03.011

• Backpropagation (BP)
• Feedforward neural networks (FNNs)
• Local minima
• Competitive learning
• Classification

The backpropagation (BP) algorithm is widely recognized as a powerful tool for training feedforward neural networks (FNNs). However, since the algorithm employs the steepest descent technique to adjust the network weights, it suffers from a slow convergence rate and often produces suboptimal solutions, which are the two major drawbacks of BP. This paper proposes a modified BP algorithm which can remarkably alleviate the problem of local minima confronted with by the standard BP (SBP). As one output of the modified training procedure, a bucket of all the possible solutions of weights matrices found during training is acquired, among which the best solution is chosen competitively based upon their performances on a validation dataset. Simulations are conducted on four benchmark classification tasks to compare and evaluate the classification performances and generalization capabilities of the proposed modified BP and SBP.

• 出版日期2012-10-1
• 单位南京大学