Average Contrastive Divergence for Training Restricted Boltzmann Machines

作者：Ma, Xuesi^*; Wang, Xiaojie

来源：Entropy, 2016, 18(1): 35.

DOI：10.3390/e18010035

• restricted Boltzmann machines
• contrastive divergence
• log-likelihood
• gradient method
• average contrastive divergence

This paper studies contrastive divergence (CD) learning algorithm and proposes a new algorithm for training restricted Boltzmann machines (RBMs). We derive that CD is a biased estimator of the log-likelihood gradient method and make an analysis of the bias. Meanwhile, we propose a new learning algorithm called average contrastive divergence (ACD) for training RBMs. It is an improved CD algorithm, and it is different from the traditional CD algorithm. Finally, we obtain some experimental results. The results show that the new algorithm is a better approximation of the log-likelihood gradient method and outperforms the traditional CD algorithm.

• 出版日期2016-1
• 单位北京邮电大学; 河南理工大学