A novel multimodal retrieval model based on ELM

作者：Zhang, Yu^*; Yuan, Ye; Wang, Yishu; Wang, Guoren

来源：Neurocomputing, 2018, 277: 65-77.

DOI：10.1016/j.neucom.2017.03.095

• Extreme Learning Machine
• Multimodal
• Probabilistic Latent Semantic Analysis
• Single hidden-layer feedforward neural networks
• Modality
• Regression
• Multimedia

In this paper, we propose a novel multimodal retrieval model based on the Extreme Learning Machine (ELM). We exploit two multimedia modalities, the image and text, to achieve the multimodal retrieval. To begin with, we employ the probabilistic Latent Semantic Analysis (pLSA) to respectively simulate the generating processes of texts and images. So we obtain the appropriate representations of the images and those of the texts. Furthermore, ELM is used for training the correlation between the representations of the images and those of the texts. So the multimodal retrieval is implemented by the learned single-hidden layer feedforward neural networks (SLFNs). Additionally, the binary classifiers are trained to improve the accuracy of the multimodal retrieval model. This multimodal model can easily be extended into other modalities and extensive experimental results demonstrate the effectiveness and efficiency of this model based on ELM.

• 出版日期2018-2-14
• 单位东北大学