A Sparse Plus Low-Rank Exponential Language Model for Limited Resource Scenarios

作者：Hutchinson Brian^*; Ostendorf Mari; Fazel Maryam

来源：IEEE/ACM Transactions on Audio Speech and Language Processing, 2015, 23(3): 494-504.

DOI：10.1109/TASLP.2014.2379593

• Language model
• log bilinear
• exponential
• sparse
• low-hyphen

This paper describes a new exponential language model that decomposes the model parameters into one or more low-rank matrices that learn regularities in the training data and one or more sparse matrices that learn exceptions (e.g., keywords). The low-rank matrices induce continuous-space representations of words and histories. The sparse matrices learn multi-word lexical items and topic/domain idiosyncrasies. This model generalizes the standard l(1)-regularized exponential language model, and has an efficient accelerated first-order training algorithm. Language modeling experiments show that the approach is useful in scenarios with limited training data, including low resource languages and domain adaptation.

• 出版日期2015-3