An active learning framework for semi-supervised document clustering with language modeling

作者：Huang Ruizhang^*; Lam Wai

来源：Data & Knowledge Engineering, 2009, 68(1): 49-67.

DOI：10.1016/j.datak.2008.08.008

• Document clustering
• Semi-supervised
• Active learning
• Language modeling

This paper investigates a framework that actively selects informative document pairs for obtaining user feedback for semi-supervised document clustering. A gain-directed document pair selection method that measures how much we can learn by revealing judgments of selected document pairs is designed. We use the estimation of term co-occurrence probabilities as a clue for finding informative document pairs. Term co-occurrence probabilities are considered in the semi-supervised document clustering process to capture term-to-term dependence relationships. In the semi-supervised document clustering, each cluster is represented by a language model. We have conducted extensive experiments on several real-world corpora. The results demonstrate that our proposed framework is effective.

• 出版日期2009-1
• 单位香港理工大学; 香港中文大学