A gesture recogintion architecture for Arabic sign language communication system

作者：Aujeszky Tamas; Eid Mohamad^*

来源：Multimedia Tools and Applications, 2016, 75(14): 8493-8511.

DOI：10.1007/s11042-015-2767-2

• Multimedia Systems and Tools
• Sign language Communication Systems
• Gesture recognition
• Avatar animation
• Usability study

Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

• 出版日期2016-7