Accuracy vs. Availability Heuristic in Multimodal Affect Detection in the Wild

作者：Bosch Nigel^*; Chen Huili; Baker Ryan; Shute Valerie; D'Mello Sidney

来源：2015 ACM International Conference on Multimodal Interaction, 2015-11-09 To 2015-11-13.

DOI：10.1145/2818346.2820739

• Missing data
• Affect
• Affect detection
• Facial expressions
• Interaction

This paper discusses multimodal affect detection from a fusion of facial expressions and interaction features derived from students' interactions with an educational game in the noisy real-world context of a computer-enabled classroom. Log data of students' interactions with the game and face videos from 133 students were recorded in a computer-enabled classroom over a two day period. Human observers live annotated learning-centered affective states such as engagement, confusion, and frustration. The face-only detectors were more accurate than interaction-only detectors. Multimodal affect detectors did not show any substantial improvement in accuracy over the face-only detectors. However, the face-only detectors were only applicable to 65% of the cases due to face registration errors caused by excessive movement, occlusion, poor lighting, and other factors. Multimodal fusion techniques were able to improve the applicability of detectors to 98% of cases without sacrificing classification accuracy. Balancing the accuracy vs. applicability tradeoff appears to be an important feature of multimodal affect detection.

• 出版日期2015
• 单位南宁师范大学