Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features

作者：Ullah, Amin; Ahmad, Jamil; Muhammad, Khan; Sajjad, Muhammad; Baik, Sung Wook^*

来源：IEEE Access, 2018, 6: 1155-1166.

DOI：10.1109/ACCESS.2017.2778011

• Action recognition
• deep learning
• recurrent neural network
• deep bidirectional long short-term memory
• convolution neural network

Recurrent neural network (RNN) and long short-term memory (LSTM) have achieved great success in processing sequential multimedia data and yielded the state-of-the-art results in speech recognition, digital signal processing, video processing, and text data analysis. In this paper, we propose a novel action recognition method by processing the video data using convolutional neural network (CNN) and deep bidirectional LSTM (DB-LSTM) network. First, deep features are extracted from every sixth frame of the videos, which helps reduce the redundancy and complexity. Next, the sequential information among frame features is learnt using DB-LSTM network, where multiple layers are stacked together in both forward pass and backward pass of DB-LSTM to increase its depth. The proposed method is capable of learning long term sequences and can process lengthy videos by analyzing features for a certain time interval. Experimental results show significant improvements in action recognition using the proposed method on three benchmark data sets including UCF-101, YouTube 11 Actions, and HMDB51 compared with the state-of-the-art action recognition methods.

• 出版日期2018