A novel probabilistic feature selection method for text classification

作者：Uysal Alper Kursat^*; Gunal Serkan

来源：Knowledge-Based Systems, 2012, 36: 226-235.

DOI：10.1016/j.knosys.2012.06.005

• Feature selection
• Filter
• Pattern recognition
• Text classification
• Dimension reduction

High dimensionality of the feature space is one of the most important concerns in text classification problems due to processing time and accuracy considerations. Selection of distinctive features is therefore essential for text classification. This study proposes a novel filter based probabilistic feature selection method, namely distinguishing feature selector (DFS), for text classification. The proposed method is compared with well-known filter approaches including chi square, information gain, Gini index and deviation from Poisson distribution. The comparison is carried out for different datasets, classification algorithms, and success measures. Experimental results explicitly indicate that DFS offers a competitive performance with respect to the abovementioned approaches in terms of classification accuracy, dimension reduction rate and processing time.

• 出版日期2012-12