A Multiobjective Learning and Ensembling Approach to High-Performance Speech Enhancement With Compact Neural Network Architectures

作者：Wang, Qing; Du, Jun^*; Dai, Li-Rong; Lee, Chin-Hui

来源：IEEE/ACM Transactions on Audio Speech and Language Processing, 2018, 26(7): 1181-1193.

DOI：10.1109/TASLP.2018.2817798

• Speech enhancement (SE)
• deep neural network (DNN)
• multiobjective learning
• multiobjective ensembling
• compact and low-latency design

In this study, we propose a novel deepneural network (DNN) architecture for speech enhancement (SE) via a multiobjective learning and ensembling (MOLE) frameworkto achieve a compact and lowlatency design, while maintaining good performance in quality evaluations. MOLE follows the boosting concept when combining weak models into a strong classifier and consists of two compact DNNs. The first, called the multiobjective learning DNN (MOL-DNN), takes multiple features, such as log-power spectra (LPS), mel-frequency cepstral coefficients (MFCCs) and Gammatone frequency cepstral coefficients (GFCCs) to predict a multiobjective set that includes clean speech feature, dynamic noise feature, and ideal ratio mask (IRM). The second, called the multiobjective ensembling DNN (MOE-DNN), takes the learned features from MOL-DNN as inputs and separately predicts clean LPS and IRM, clean MFCC and IRM, and clean GFCC and IRM using three sets of weak regression functions. Finally, a post-processing operation can be applied to the estimated clean features by leveraging the multiple targets learned from both the MOLDNN and the MOE-DNN. On speech corrupted by 15 noise types not seen in model training the SE results show that the MOLE approach, which features a small model size and low run-time latency, can achieve consistent improvements over both DNN- and long short-term memory (LSTM)-based techniques in terms of all the objective metrics evaluated in this study for all three cases (the input contexts contain 1-frame, 4-frame and 7-frame instances). The 1-frameMOLE-based SE system outperforms the DNN-based SE system with a 7-frame input expansion at a 3-frame delay and also achieves better performance than the LSTM-based SE system with 4-frame, no delay expansion by including only 3 previous frames, and with 170 times less processing latency.

• 出版日期2018-7
• 单位中国科学技术大学