A simple probabilistic explanation of term frequency-inverse document frequency (tf-idf) heuristic (and variations motivated by this explanation)

作者：Havrlant Lukas; Kreinovich Vladik^*

来源：International Journal of General Systems, 2017, 46(1): 27-36.

DOI：10.1080/03081079.2017.1291635

• term frequency-inverse document frequency (tf-idf)
• probabilistic explanation

In document analysis, an important task is to automatically find keywords which best describe the subject of the One of the most widely used techniques for keyword detection is a technique based on the term frequency-inverse document frequency (tf-idf) heuristic. This techniques has some explanations, but these explanations are somewhat too complex to be fully convincing. In this paper, we provide a simple probabilistic explanation for the tf-idf heuristic. We also show that the ideas behind explanation can help us come up with more complex formulas which will hopefully lead to a more adequate detection of keywords.

• 出版日期2017