A Strategy on Selecting Performance Metrics for Classifier Evaluation

作者：Liu, Yangguang; Zhou, Yangming; Wen, Shiting; Tang, Chaogang

来源：International Journal of Mobile Computing and Multimedia Communications, 2014, 6(4): 20-35.

DOI：10.4018/IJMCMC.2014100102

• Classification accuracy
• Classification models
• Classifier's performance
• Machine learning communities
• Performance metrics
• Receiver operating curves
• Root mean square errors
• Spearman rank correlation

The evaluation of classifiers'performances plays a critical role in construction and selection of classification model. Although many performance metrics have been proposed in machine learning community, no general guidelines are available among practitioners regarding which metric to be selected for evaluating a classifier's performance. In this paper, we attempt to provide practitioners with a strategy on selecting performance metrics for classifier evaluation. Firstly, the authors investigate seven widely used performance metrics, namely classification accuracy, F-measure, kappa statistic, root mean square error, mean absolute error, the area under the receiver operating curve, and the area under the precision-recall curve. Secondly, the authors resort to using Pearson linear correlation and Spearman rank correlation to analyses the potential relationship among these seven metrics. Experimental results show that these commonly used metrics can be divided into three groups, and all metrics within a given group are highly correlated but less correlated with metrics from different groups.

• 出版日期2014-10-1
• 单位浙江大学; 中国矿业大学（北京）