Adaptive crawler for external hyperlinks search and acquisition

作者:Pechnikov A A*; Chernobrovkin D I
来源:Automation and Remote Control, 2014, 75(3): 587-593.
DOI:10.1134/S0005117914030151

摘要

We describe a search robot (crawler) intended to collect information regarding outgoing hyperlinks from a given set of web sites related to a certain topic. The crawler%26apos;s adaptive behavior is formulated in terms of a multi-armed bandit problem. Our experiments show that the choice of an adaptive algorithm for the crawler%26apos;s rational behavior depends on the actual topic of the underlying set of web sites.

  • 出版日期2014-3

全文