A New Data Layout Scheme for Energy-Efficient MapReduce Processing Tasks

作者：Tran Xuan T; Tien Van Do^*; Rotter Csaba; Hwang Dosam

来源：Journal of Grid Computing, 2018, 16(2): 285-298.

DOI：10.1007/s10723-018-9433-7

• Hadoop
• Layout
• Big data processing
• Data locality
• Dynamic power management

Yet Another Resource Negotiator (YARN) is a framework to manage and allocate resource requests from applications that process big data stored in HDFS. However, dynamic power management methods are not efficient when YARN manage applications to process big data stored in the default data layout of HDFS. In this paper, we propose a new data layout scheme that can be implemented for HDFS. A comparison between our proposal and the existing HDFS data layout scheme shows that the new data layout algorithm significantly reduces the energy consumption at the slight expense of the mean response time of jobs.

• 出版日期2018-6