Hybrid Parallel Inference for Hierarchical Dirichlet Processes

作者：Omoto Tsukasa^*; Eguchi Koji; Tora Shotaro

来源：IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2014, E97D(4): 815-820.

DOI：10.1587/transinf.E97.D.815

• hierarchical dirichlet process
• topic models
• parallelization

The hierarchical Dirichlet process (HDP) can provide a nonparametric prior for a mixture model with grouped data, where mixture components are shared across groups. However, the computational cost is generally very high in terms of both time and space complexity. Therefore, developing a method for fast inference of HDP remains a challenge. In this paper, we assume a symmetric multiprocessing (SMP) cluster, which has been widely used in recent years. To speed up the inference on an SMP cluster, we explore hybrid two-level parallelization of the Chinese restaurant franchise sampling scheme for HDP, especially focusing on the application to topic modeling. The methods we developed, Hybrid-AD-HDP and Hybrid-Diff-AD-HDP, make better use of SMP clusters, resulting in faster HDP inference. While the conventional parallel algorithms with a full message-passing interface does not benefit from using SMP clusters due to higher communication costs, the proposed hybrid parallel algorithms have lower communication costs and make better use of the computational resources.

• 出版日期2014-4