Non-Intrusive Anomaly Detection With Streaming Performance Metrics and Logs for DevOpsin Public Clouds: A Case Study in AWS

作者：Sun, Daniel^*; Fu, Min; Zhu, Liming; Li, Guoqiang; Lu, Qinghua

来源：IEEE Transactions on Emerging Topics in Computing, 2016, 4(2): 278-289.

DOI：10.1109/TETC.2016.2520883

• Public clouds
• non-intrusive anomaly detection
• performance metrics
• logs
• machine learning

Public clouds are a style of computing platforms, where scalable and elastic Information Technology-enabled capabilities are provided as a service to external customers using Internet technologies. Using public cloud services can reduce costs and increase the choices of technologies, but it also implies limited system information for users. Thus, anomaly detection at user end has to be non-intrusive and hence difficult, particularly during DevOps operations because the impacts from both anomalies and these operations are often indistinguishable, and hence, it is hard to detect the anomalies. In this paper, our work is specific to a successful public cloud, Amazon Web Service, and a representative DevOps operation, rolling upgrade, on which we report our anomaly detection that can effectively detect anomalies. Our anomaly detection requires only metrics data and logs supplied by most public clouds officially. We use support vector machine to train multiple classifiers from monitored data for different system environments, on which the log information can indicate the best suitable classifier. Moreover, our detection aims at finding anomalies over every time interval, called window, such that the features include not only some indicative performance metrics but also the entropy and the moving average of metrics data in each Our experimental evaluation systematically demonstrates the effectiveness of our approach.

• 出版日期2016-4
• 单位中国石油大学（华东）; 中国石油大学（北京）; 上海交通大学