A Bidirectional LSTM Approach with Word Embeddings for Sentence Boundary Detection

作者：Xu, Chenglin^*; Xie, Lei; Xiao, Xiong

来源：Journal of Signal Processing Systems for Signal Image and Video Technology, 2018, 90(7): 1063-1075.

DOI：10.1007/s11265-017-1289-8

• Sentence boundary detection
• Word embedding
• Recurrent neural network
• Long short-term memory

Recovering sentence boundaries from speech and its transcripts is essential for readability and downstream speech and language processing tasks. In this paper, we propose to use deep recurrent neural network to detect sentence boundaries in broadcast news by modeling rich prosodic and lexical features extracted at each inter-word position. We introduce an unsupervised word embedding to represent word identity, learned from the Continuous Bag-of-Words (CBOW) model, into sentence boundary detection task as an effective feature. The word embedding contains syntactic information that is essential for this detection task. In addition, we propose another two low-dimensional word embeddings derived from a neural network that includes class and context information to represent words by supervised learning: one is extracted from the projection layer, the other one comes from the last hidden layer. Furthermore, we propose a deep bidirectional Long Short Term Memory (LSTM) based architecture with Viterbi decoding for sentence boundary detection. Under this framework, the long-range dependencies of prosodic and lexical information in temporal sequences are modeled effectively. Compared with previous state-of-the-art DNN-CRF method, the proposed LSTM approach reduces 24.8% and 9.8% relative NIST SU error in reference and recognition transcripts, respectively.

• 出版日期2018-7
• 单位西北工业大学; 南阳理工学院