Unsupervised Visual and Textual Information Fusion in CBMIR Using Graph-Based Methods

作者：Ah Pine Julien^*; Csurka Gabriela; Clinchant Stephane

来源：ACM Transactions on Information Systems, 2015, 33(2): 9.

DOI：10.1145/2699668

• Algorithms
• Theory
• Content-based multimedia information retrieval
• information fusion
• graph-based methods
• cross-media similarity
• random walk
• Visual reranking

Multimedia collections are more than ever growing in size and diversity. Effective multimedia retrieval systems are thus critical to access these datasets from the end-user perspective and in a scalable way. We are interested in repositories of image/text multimedia objects and we study multimodal information fusion techniques in the context of content-based multimedia information retrieval. We focus on graph-based methods, which have proven to provide state-of-the-art performances. We particularly examine two such methods: cross-media similarities and random-walk-based scores. From a theoretical viewpoint, we propose a unifying graph-based framework, which encompasses the two aforementioned approaches. Our proposal allows us to highlight the core features one should consider when using a graph-based technique for the combination of visual and textual information. We compare cross-media and random-walk-based results using three different real-world datasets. From a practical standpoint, our extended empirical analyses allow us to provide insights and guidelines about the use of graph-based methods for multimodal information fusion in content-based multimedia information retrieval.

• 出版日期2015-2