Acceleration of Neural Network Learning by GPGPU

作者:Tsuchida Yuta*; Yoshioka Michifumi
来源:Electronics and Communications in Japan, 2013, 96(8): 59-66.
DOI:10.1002/ecj.11412

摘要

Recently, graphic boards have come to have higher performance than CPUs as a result of the development of 3DCG and movie processing, and are now widely used due to progress in computer entertainment. Implementation of general-purpose computing on GPU (GPGPU) has become easier as a result of the integrated development environment CUDA distributed by NVIDIA. A GPU has dozens or hundreds of arithmetic circuits, whose allocations are controlled by CUDA. In prior research, the implementation of a neural network using GPGPU has been studied; however, the training of networks was not mentioned because the GPU performance is low in conditional processing but high in linear algebra processing. Therefore, we have proposed two methods. First, a whole network is implemented as a thread, and some networks are trained in parallel to shorten the time necessary to find the optimal weight coefficients. Second, this paper introduces parallelization in the neural network structure, in which the calculation of neurons in the same layers is parallelized. The processing to train the same network with different patterns is likewise independent. As a result, the second method is 20 times faster than the CPU, and 6 times as fast as the first proposed method.

  • 出版日期2013-8

全文