An Efficient Distributed Programming Model for Mining Useful Patterns in Big Datasets

作者：Karim Md Rezaul^*; Ahmed Chowdhury Farhan; Jeong Byeong Soo; Choi Ho Jin

来源：IETE Technical Review, 2013, 30(1): 53-63.

DOI：10.4103/0256-4602.107340

• Associated pattern
• Business-oriented data
• ComMapReduce
• Correlated pattern
• Independent pattern
• Null transactions
• Transactional database

Mining combined association rules with correlation and market basket analysis can discover customers buying purchase rules along with frequently correlated, associated-correlated, and independent patterns synchronously which are extraordinarily useful for making everydays business decisions. However, due to the main memory bottleneck in single computing system, existing approaches fail to handle big datasets. Moreover, most of them cannot overcome the screenings and overhead of null transactions; hence, performance degrades drastically. In this paper, considering these limitations, we propose a distributed programming model for mining business-oriented transactional datasets by using an improved MapReduce framework on Hadoop, which overcomes not only the single processor and main memory-based computing, but also highly scalable in terms of increasing database size. Experimental results show that the technique proposed and developed in this paper are feasible for mining big transactional datasets in terms of time and scalability.

• 出版日期2013-2