A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling

作者：Zhang, Qingchen; Lin, Man; Yang, Laurence T.^*; Chen, Zhikui; Khan, Samee U.; Li, Peng

来源：IEEE Transactions on Services Computing, 2019, 12(5): 739-749.

DOI：10.1109/TSC.2018.2867482

• Heuristic algorithms
• Computational modeling
• Edge computing
• Task analysis
• Energy consumption
• Cloud computing
• Internet of Things
• energy saving
• deep Q-learning
• dynamic voltage and frequency scaling
• rectified linear units

Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average $2\%\hbox{-}2.4\%$2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.

• 出版日期2019-10
• 单位电子科技大学; 大连理工大学