A family of multi-rater kappas that can always be increased and decreased by combining categories

作者：Warrens Matthijs J^*

来源：Statistical Methodology, 2012, 9(3): 330-340.

DOI：10.1016/j.stamet.2011.08.008

• Inter-rater reliability
• Cohen%26apos
• s kappa
• Schouten-type inequality
• Hubert%26apos
• s kappa
• Mielke, Berry and Johnston%26apos
• s kappa

Cohen%26apos;s kappa is a popular descriptive statistic for measuring agreement between two raters on a nominal scale. Various authors have generalized Cohen%26apos;s kappa to the case of m %26gt;= 2 raters. We consider a family of multi-rater kappas that are based on the concept of g-agreement (g = 2, 3 m), which refers to the situation in which it is decided that there is agreement if g out of m raters assign an object to the same category. For the family of multi-rater kappas we prove the following existence theorem: In the case of three or more categories there exists for each multi-rater kappa kappa (M. g) two categories such that, when combined, the kappa (m. g) value increases. In addition, there exist two categories such that, when combined, the kappa (m, g) value decreases.

• 出版日期2012-5