Adaptive application-aware job scheduling optimization strategy in heterogeneous infrastructures

作者：Kim Seoyoung; Choi Jieun; Kim Yoonhee^*

来源：Cluster Computing, 2016, 19(3): 1515-1526.

DOI：10.1007/s10586-016-0588-3

• Job scheduling optimization
• Adaptive evaluation
• Application-aware
• High-throughput computing

As the rapid advancement and diversity in the computing systems, it is demanding to take the most robust scheduling algorithm that guarantee an optimized performance to execute manifold applications on large-scale heterogeneous computing environments. This paper present an adaptive application-aware job scheduling optimization strategy for large-scale high throughput computing in heterogeneous infrastructures. The proposed scheduling optimization method is built on two main concepts. First, it provides application-aware job distribution weights through empirical data in large-scale heterogeneous infrastructures. Here we adopt the concept of weight, which represent the ratio of tasks that will be computed on each resource. The weights can vary in terms of application type and are optimized until the system get in steady status. Secondly, it offers an adaptive control phase that is invoked by the weight adjustment and resource scaling feature. The feedback data from monitoring module is forwarded to the control phase in order to adjust weights and over-provisioning ratio, and result in enhancing overall balance between performances and utilization of system. The experimental evaluation with the four realistic workload patterns demonstrates that, when compared to the core-based scheme which distributes tasks in proportions of each resource's number of cores, the use of our optimization method can achieve 62 % better average throughput, 43 % shorter average queueing time, and 38 % better average utilization of the entire resources in diverse infrastructure environments by harnessing our adaptive module.

• 出版日期2016-9