Discriminative reranking approach to spelling correction

作者：Zhang Yang^*; He Pi Lian; Xiang Wei; Li Mu

来源：Journal of Software, 2008, 19(3): 557-564.

DOI：10.3724/SP.J.1001.2008.00557

• Art state techniques
• Aspell
• Discriminative model
• Edit distance
• Log mining
• Microsoft word 2003
• Noisy channel model
• Phonetic similarity
• Query chain
• Ranking SVM
• Spelling correction

This paper proposes an approach to spelling correction. It reranks the output of an existing spelling corrector, Aspell. A discriminative model (Ranking SVM) is employed to improve upon the initial ranking, using additional features as evidence. These features are derived from state-of-the-art techniques in spelling correction, including edit distance, letter-based n-gram, phonetic similarity and noisy channel model. This paper also presents a method to automatically extract training samples from the query log chain. The system outperforms the baseline Aspell greatly, as well as the previous models and several off-the-shelf systems (e.g. spelling corrector in Microsoft Word 2003). The experimental results based on query chain pairs are comparable to that based on manually-annotated pairs, with 32.2%/32.6% reduction in error rate, respectively.

• 出版日期2008
• 单位天津大学