A Dynamic Rule Creation Based Anomaly Detection Method for Identifying Security Breaches in Log Records

作者：Breier Jakub^*; Branisova Jana

来源：Wireless Personal Communications, 2017, 94(3): 497-511.

DOI：10.1007/s11277-015-3128-1

• Network security
• Log analysis
• Anomaly detection
• Data mining
• Apache Hadoop
• MapReduce

Evidence of security breaches can be found in log files, created by various network devices in order to provide information about their operation. Huge amount of data contained within these files usually prevents to analyze them manually, therefore it is necessary to utilize automatic methods capable of revealing potential attacks. In this paper we propose a method for anomaly detection in log files, based on data mining techniques for dynamic rule creation. To support parallel processing, we employ Apache Hadoop framework, providing distributed storage and distributed processing of data. Outcomes of our testing show potential to discover new types of breaches and plausible error rates below 10 %. Also, rule generation and anomaly detection speeds are competitive to currently used algorithms, such as FP-growth and apriori.

• 出版日期2017-6