A LEMPEL-ZIV COMPLEXITY-BASED NEURAL NETWORK PRUNING ALGORITHM

作者：Ahmed Sultan Uddin^*; Shahjahan Md; Murase Kazuyuki

来源：International Journal of Neural Systems, 2011, 21(5): 427-441.

DOI：10.1142/S0129065711002936

• Artificial Neural Networks (ANNs)
• pruning
• ANN construction
• complexity of ANN
• Lempel-Ziv complexity

This paper presents a pruning method for artificial neural networks (ANNs) based on the 'Lempel-Ziv complexity' (LZC) measure. We call this method the 'silent pruning algorithm' (SPA). The term 'silent' is used in the sense that SPA prunes ANNs without causing much disturbance during the network training. SPA prunes hidden units during the training process according to their ranks computed from LZC. LZC extracts the number of unique patterns in a time sequence obtained from the output of a hidden unit and a smaller value of LZC indicates higher redundancy of a hidden unit. SPA has a great resemblance to biological brains since it encourages higher complexity during the training process. SPA is similar to, yet different from, existing pruning algorithms. The algorithm has been tested on a number of challenging benchmark problems in machine learning, including cancer, diabetes, heart, card, iris, glass, thyroid, and hepatitis problems. We compared SPA with other pruning algorithms and we found that SPA is better than the 'random deletion algorithm' (RDA) which prunes hidden units randomly. Our experimental results show that SPA can simplify ANNs with good generalization ability.

• 出版日期2011-10