SparseConnect: regularising CNNs on fully connected layers

作者：Xu, Qi; Pan, Gang^*

来源：Electronics Letters, 2017, 53(18): 1246-1247.

DOI：10.1049/el.2017.2621

• Benchmark datasets
• Generalisation
• Over fitting problem
• Over-fitting training
• Overfitting
• Regularisation
• State of the art

Deep convolutional neural networks (CNNs) have achieved unprecedented success in many domains. The numerous parameters allow CNNs to learn complex features, but also tend to hinder generalisation by over-fitting training data. Despite many previously proposed regularisation methods, over-fitting is one of many problems in training a robust CNN. Among many factors that may lead to over-fitting, the numerous parameters of fully connected layers (FCLs) of a typical CNN are a contributor to the over-fitting problem. The authors propose SparseConnect, which alleviates over-fitting by sparsifying connections to FCLs. Experimental results on three benchmark datasets MNIST and CIFAR10 show SparseConnect outperforms several state-of-the-art regularisation methods.

• 出版日期2017-8-31
• 单位浙江大学